Dortmund opened the scoring early in the 11th minute when Marco Russ scored in his own net.
Batshuayi quickly restored Dortmund's lead just 15 minutes after his introduction and the Belgian netted a last-gasp victor having seen Danny Blum level matters just three minutes earlier in stoppage time.
BVB pulls into third place, a point behind rivals Schalke, and leaves Eintracht Frankfurt in fifth.
But Batshuayi, dropped to the bench after five games without a goal in all competitions, could not argue with Stoger's decision to leave him out.More news: Kawhi Leonard: Could return vs. Pelicans
"We lost today because Dortmund scored one more goal than we did", Boateng said.
But Batshuayi wasn't finished and took advantage of a exhausted Frankfurt offside trap to score an even later victor and give Dortmund an invaluable three points in the battle to qualify for the Champions League. Reus, who extended his Dortmund contract on Friday, would have nearly certainly scored either way.
Resurgent in the second half, Frankfurt finally found an equaliser on 75 minutes.
The Belgium worldwide, on loan at BVB from Chelsea, was dropped to the bench for the Bundesliga home game after five games without a goal in all competitions.
Kevin-Prince Boateng almost scored with a clever backheel just after half-time, but was denied by Dortmund goalkeeper Roman Burki. The Serbian forward got his head to Jonathan de Guzman's free kick.More news: Church of Scientology to launch TV network
It's a refreshing change for supporters who have grown exhausted of seeing their club repeatedly sell its best players in recent years, but local hero Reus said he wanted to "show both the club and my teammates that something can develop here". The Frankfurt goalkeeper's contract is up at the end of the season and has been linked as a replacement for the retiring Roman Weidenfeller.
"I hope it's a signal", he said.
Timo Werner had the best chances for Leipzig on his return to his former side.
No victor could be found when Stuttgart and RB Leipzig played to a goalless draw at the Mercedes-Benz Arena.
Leipzig, which hasn't won in four games, remains sixth, four points behind Leverkusen. The Germany striker missed a great opportunity before the break when Stuttgart 'keeper Ron-Robert Zieler got a foot to his shot.More news: Michigan, MSU both No 3. seeds in NCAA Tournament
- Warren Gatland celebrates 'a job done' as Wales conquer Italy
- OnePlus 5/5T to get Android 8.1 update soon
- MSU earns 3 seed in Midwest Region of NCAA Tournament
- Kyrie Irving Exits Game With Knee Soreness
- 'October' team to connect with fans live on trailer launch day
- 'Armed' PAOK owner invades pitch, confronts referee after late confusion
- Scam and stealing offences, Brisbane and Gold Coast
- Intel casts doubt on report of interest in Broadcom
- Japan ministry admits doctoring documents linked to PM's wife
- Forecast: Rain develops tonight; a chance for snow late Monday