Hale Leys United were grateful to former Aylesbury United striker Roni Joe after he rescued them from Berks and Bucks Cup humiliation at the weekend.
Leys made short trip to Bierton for their second round match with a slightly changed squad due to injuries.
Hale Leys, who have won six of their seven games this season, went into the game as red hot favourites but Bierton had other ideas and managed to defend their goal until half-time.
They went one step further in the second half as Adam Mellis scored for Bierton.
Leys threw the kitchen sink at Bierton in an effort to stay in the competition but it took substitute Joe to equalise at the death in the 85th minute to take it to extra-time.
Leys then went ahead as Tom Brindley made it 2-1 and Joe added a third before the last 15 minutes of time added on to make things more comfortable.
This Saturday Hale Leys entertain Mursley (3pm kick off) at their Fairford Leys home ground and the club is urging please come down and show your support.