Rational numbers in $\times b$-invariant sets

Let $b \geq 2$ be an integer and $S$ be a finite non-empty set of primes not containing divisors of $b$. For any non-dense set $A \subset [0,1)$ such that $A \cap \mathbb{Q}$ is invariant under $\times b$ operation, we prove the finiteness of rational numbers in $A$ whose denominators can only be divided by primes in $S$. A quantitative result on the largest prime divisors of the denominators of rational numbers in $A$ is also obtained.