The fitment from upper to lower has negligible affects to accuracy downrange compared to how your barrel and handguard fit together, or the use of a bipod on the handguard. Multiple folks shooting benchrest and f-class will affirm that 1) the size of the upper receiver at the barrel extension junction, and 2) the better the handguard/barrel attachments bring about more accuracy issues than how well the upper and lower fit together. Focus on good quality barrel, quality upper receiver, and you'll satisfy majority of your accuracy issues. The handguard diameter and how it mounts to the upper is going to also assist with this concern.
It's more annoying and less "custom" if they shift, but that's about it.