Ants can cause a lot of unsightliness to your paver project or any pavement that has joints, cracks, or control joints. They are a nuisance to deal with and to remove from your paver patio, driveway, or walkway. However, there are ways in which you can install your project and the product choices to not have to deal with them as much.
But if ants are prevalent in your area, they will appear in any cracks in pavement such as control joints in concrete or cracks in concrete and asphalt. There is no way around this.

Ants Under Pavers
Ants will burrow and build their colony where there are a lot of fine materials that help them thrive. With most paver bases and joint fills, they have this fine material available to them to build their colonies which makes it the perfect environment for them to thrive. There are ways around this where you can build your project to help reduce their ability to thrive, but for traditionally prepared paver bases and joint fills they have the perfect material to build their colonies.
If you know that your area has prevalence of ants and know it could present a problem, there are ways in which you can prepare your base and choose the correct joint fill to reduce their ability to thrive under your pavers.
How to Stop Ants Under Pavers
If there are ants in your patio pavers, then you will be in reaction mode to get rid of these ants. The same methods of removing ants from your house can be applied to the ants in your pavers. Traps and spray can be used, but be careful with potentially staining your pavers with these. The next option would be to lift those pavers, remove the ants, and re-lay those pavers. Not a great option as it involves more work and it does not fix the problem from occurring again as this has to do with the way the paver project was built in the first place.
A lot of this can be mitigated with the right products in your base preparation and joint fill to prevent ants in the first place. If you opt for a dense graded base for your pavers which includes a Granular A (3/4″ crushed stone down to fines) and a Concrete Sand bedding layer, you are giving ants exactly what they need and want to be able to build their colony. They will use those fine granules in that base material and thrive.
Instead, try using an open graded base which consists of 3/4″ crushed clear stone (no fines) and a bedding layer of HPB or 1/4″ clean chip which is also clear of fines. These aggregates are large enough that ants will not be able to carry around and build their colony. You can also use a synthetic base which includes paver base panels on top of HPB which would also help in deterring ants from building in.
There still remains the joint fill aspect of the project which typically will always include some sort of finer granule. Polymeric sand will harden and prevent ants. But the keyword there is prevent and not eliminate the problem of ants. Over time, that sand will loosen up and need to be replaced. If not done in a timely manner, it will break down and ants will like to use those finer granules in the polymeric sand to build their colony. You can add a sealer to polymeric sand a minimum of 30 days after the installation of that sand to help further prevent ants taking over, but it still remains something that you will need to maintain in order to keep the ants away. Both the polymeric sand and the sealer would need to be maintained in this case.
You can opt for a semi-permeable resin-based jointing compound which solidifies and in our experience remains that way much longer than polymeric sand. These compounds still include finer granules, but with it remaining solid it is difficult for ants to break this material down. These do need to be installed with an open graded or synthetic base for the pavers as they will introduce water through their permeability into that base material and down into the subsoil. They cannot be used in a dense graded application.

