Just put all resources into simulating dirt well, then make a game about driving various power equipment. A sandbox game, where you just build whatever you want. VR would be fun.
Actually, I was recently thinking of gravel pit simulator. It's be like a farming sim, you'd buy various equipment and use them to move dirt separating gravel and selling it to get money to buy better requirements.
That’s be great. It would be especially cool if you could simulate gravel, with all its friction and inertia and everything. And then somehow make that same model handle everything down to the molecular sheets of clay.
The game 'Captain of Industry' isn't a simulator, but it did come to mind when reading this comment. Kinda like cities skylines but with a heavy focus on excavation. There's one map where I spent hours excavating a path up a mountainside to escape the starting area.