The former York City hitman netted a 56th-minute penalty and then fired home a winner right at the end to give Knill's struggling team plenty to smile about.
"Andy was fantastic for us, he took both of his goals really well," said Knill. "It was a fantastic goal to win the game. Bish put in a good shift for us.
"There's a lot of speculation about Andy at the moment, but when he's on the pitch he works hard.
"My first win in charge of the club feels fantastic, we thoroughly deserved our victory.
"It was a good team performance."
Earlier, Peter Thorne headed the home side into the lead from a Matthew Clarke cross.
But Knill's charges bravely battled back from behind to make sure of the points. Knill made two changes to his side beaten 3-2 at home to Stockport on Tuesday night.
Keeper Jim Provett was dropped and replaced by new loan signing Darren Randolph, while fellow new recruit Adam Rooney was given the chance to shine in place of Glynn Hurst.
Randolph saved well from Willy Topp early in the matches, while Rooney tested Bradford keeper Scott Loach with a powerful shot.
But having gone behind to Thorne's opener, Bishop led the comeback.
David Wetherall needlessly felled Ben Futcher in the box and referee Kevin Friend pointed to the spot.
Bishop's sweet left-foot strike was too good for Loach as it flew into the bottom left-hand corner.
However, the best was yet to come as mounting pressure told.
Bishop hit a screamer of a shot from the edge of the penalty area to send the travelling fans wild; a goal that could prove crucial in Bury's battle to avoid the drop from League Two.
Tweet
