Taking A/B test results at face value goes against the fundamentals of experimentation.
A few weeks ago, I was in a meeting with a very senior product manager, with about decade and a half of experience in PM. He proudly mentioned that the company runs hundreds of A/B tests and experiments every day. They take the results of the experiments and use it to tweak the product.
