A season's best 1,399 crowd saw the super Celts reach the FA Cup second round for the second time in seven years.
Those left queuing outside at the start missed two goals in a bizarre opening few minutes. The game was just 45 seconds old when Merthyr's Lee Price handled inside his own box to allow captain Gary Bauress to coolly slot home from the penalty spot.
But Celtic were ahead for barely a minute when Craig Lima headed the leveller at the near post after some slack marking by Stalybridge defenders.
Stalybridge endured some nervy moments but on 25 minutes, the impressive Steve Pickford restored their lead. He calmly picked his spot to score through a crowded penalty area after Winfield Steele's earlier shot had come back off the post.
The second half had Celtic fans on the edge of their seats as slick Merthyr searched for the elusive equaliser. But, after Kevin Parr had again rattled the woodwork, substitute Tony Sullivan hit a rasping 20-yard strike with 5 minutes left to put the game beyond doubt.