Cutting larger corners makes a box taller but its base smaller. The best cut balances both effects.
Differentiate to get V′(x) = 96 − 80x + 12x². The feasible critical point is (10 − 2√7)/3 ≈ 1.569. The endpoint volumes are zero, so this interior maximum wins.
A critical point outside the physical domain cannot solve this box problem.
The sheet is 12 × 8 units. At either endpoint, the box has zero volume.