If you have too large of a filter you can actually lose overall power.
Here's why, Velocity.
When you have a properly sized filter, the vacuum created behind the filter (TB side) is high, and when you open the throttle, it can accelerat much quicker than when the vacuum at the same point is low, slowing the incoming air, or rather the ability to gain more air as the engine needs it.