Snail mode for simulating vast databases when using a small test database

[SpontaneousOrder]

If your data is too generic then your queries may not be selective enough to use indexes and you may have performance issues you would not get in production. Or if your test data is too random then indexes may be used that would not in production.

Contention due to data distribution is an issue to be aware of too. I've build a system before that ran like treacle through a sieve in test. it turned out that they were putting through hundreds of TPS using only 50 credit card numbers. When they added more credit card numbers it flew.

[Sysman]

Sticking data on the cloud won't fly for most organisations that have large databases.

Also many companies have test and dev databases that are considerably smaller than production.

I see the relevance of what owlhoot is suggesting.

Yep. I've seen it time and again where projects have been developed on the latest flashy kit but expected to run well on lesser machines. Graphics designers using gigantic screens and bags of RAM not thinking about the poor users with their commodity gear is one example. With SQL, there's a similar danger that developers might have fancy disk arrays or SSDs to develop on but the end users have to live with existing kit which already has a workload.

"But this page worked so much faster when we tested it before the release" is not an uncommon complaint.

Run the query on prod, stick that time 'estimate' into the DEV and UAT phases and give the users a real feel on how it will perform on prod.

Sounds better.

The idea of restricting server RAM and loading up a test database on some USB2 disks has merit