For millions of years, nature has basically been getting by with just a few elements from the periodic table. Carbon, calcium, oxygen, hydrogen, nitrogen, phosphorus, silicon, sulfur, magnesium and potassium are the building blocks of almost all life on our planet (tree trunks, leaves, hairs, teeth,...
It's not like these elements are disappearing into the void. If we start running out of something, the price will increase, and we'll either find alternatives or put more effort into recycling.
For example, see the increasing use of LiFePO4 batteries, without nickel or cobalt.
The health and pollution risks unfortunately don't scale the same as the economics. We could potentially poison ourselves or destroy major biomes before certain elements "run out" in their accessible forms.
Particularly, I worry that CO₂, plastics, uranium, HCFCs etc. are just the first of many problems we'll have with breaking down these materials. The non-biological elementome will not degrade, at least not without leaving non-biological elements behind.
That can be fine. Rocks generally don't participate in the biological cycle either and they don't bother anyone.
But for example, plastics are practically rocks in funny shapes, which float out into the ocean. Even just that tiny difference causes problems for maritime wildlife. Other super-durable materials will produce different rocks, which may cause problems in new and innovative ways.
And of course, not everything we use is a rock. Some materials will genuinely just interact with our surroundings in destructive ways. The hope is that they do then degrade.
There's already been issues like this in using copper sulfate (which is inorganic in terms of molecular structure and persistent as an element like you mention, but labeled as organic in some jurisdictions in terms of agricultural treatments and food marketing) as a fungicide. It's a very short lived fungicide for leaf borne fungal diseases, meaning it must be present on the leaf to prevent infection (it is easily washed away by rain or errant irrigation) and applied repeatedly, but is long lived in the soil meaning it can build up and kill off mycorrhizae and other beneficial soil fungus causing longer term drops in yield.
So, yeah, your worry is valid. And, that's just one example.
Plastics are not rocks in funny shapes. We are made of plastics. They're just unusual compounds which no primary decomposer has developed yet.
That's not to say we shouldn't address the issue, but it's important to understand what the issue actually is. The fact that plastics are familiar yet unfamiliar compounds is actually what causes the problems.
Where do you get the idea we are made of plastics? Not necessarily throwing shade, just.. I'm a molecular biologist and at first pass that seems like a stretch. I'd be excited to be wrong
Thermosets and thermoplastics, right? Not sure that we have that going on in there...