Dortmund opened the scoring early in the 11th minute when Marco Russ scored in his own net.
Batshuayi quickly restored Dortmund's lead just 15 minutes after his introduction and the Belgian netted a last-gasp victor having seen Danny Blum level matters just three minutes earlier in stoppage time.
BVB pulls into third place, a point behind rivals Schalke, and leaves Eintracht Frankfurt in fifth.
But Batshuayi, dropped to the bench after five games without a goal in all competitions, could not argue with Stoger's decision to leave him out.More news: 'Armed' PAOK owner invades pitch, confronts referee after late confusion
"We lost today because Dortmund scored one more goal than we did", Boateng said.
But Batshuayi wasn't finished and took advantage of a exhausted Frankfurt offside trap to score an even later victor and give Dortmund an invaluable three points in the battle to qualify for the Champions League. Reus, who extended his Dortmund contract on Friday, would have nearly certainly scored either way.
Resurgent in the second half, Frankfurt finally found an equaliser on 75 minutes.
The Belgium worldwide, on loan at BVB from Chelsea, was dropped to the bench for the Bundesliga home game after five games without a goal in all competitions.
Kevin-Prince Boateng almost scored with a clever backheel just after half-time, but was denied by Dortmund goalkeeper Roman Burki. The Serbian forward got his head to Jonathan de Guzman's free kick.More news: Cowboys miss NCAA Tournament, will host NIT game
It's a refreshing change for supporters who have grown exhausted of seeing their club repeatedly sell its best players in recent years, but local hero Reus said he wanted to "show both the club and my teammates that something can develop here". The Frankfurt goalkeeper's contract is up at the end of the season and has been linked as a replacement for the retiring Roman Weidenfeller.
"I hope it's a signal", he said.
Timo Werner had the best chances for Leipzig on his return to his former side.
No victor could be found when Stuttgart and RB Leipzig played to a goalless draw at the Mercedes-Benz Arena.
Leipzig, which hasn't won in four games, remains sixth, four points behind Leverkusen. The Germany striker missed a great opportunity before the break when Stuttgart 'keeper Ron-Robert Zieler got a foot to his shot.More news: Warren Gatland celebrates 'a job done' as Wales conquer Italy
- Raid on Iran's Diplomatic Residence in Vienna: Attacker Shot Dead
- Trump pushing gun reform proposals that include age limit increase on purchases
- Kawhi Leonard: Could return vs. Pelicans
- Intel casts doubt on report of interest in Broadcom
- US officials say no added conditions for North Korea talks
- Melrose Raises Offer for GKN to $11.2 Billion to Sway Investors
- Renee Baio reveals she was diagnosed with a brain disease
- Fiorentina pay tribute to Davide Astori during win over Benevento
- MSU earns 3 seed in Midwest Region of NCAA Tournament
- Militants Killed In Encounter In Jammu And Kashmir's Anantnag: Army