Celtic, boasting a wealth of former Barrow players in their squad, were unlucky not to get more than just a point in this scrappy affair.
Barrow dominated the first half and could have taken the lead after 11 minutes when David Rush had a close range shot blocked off the line by a Celtic defender.
Stalybridge, however, took the lead on 18 minutes when Steve Pickford's in-swinging corner was met at the near post by Steve Jones.
A defensive blunder on the stroke of half-time allowed Barrow to equalise. Two Celtic players collided in the box and Micky Waller volleyed home from 12 yards.
Celtic were the better side after the break and they went close on several occasions. Keeper Phil Naisbett kept Barrow in the game with two brilliant saves as the home team hung on.