Travis Watson had 15 points and nine rebounds while five other Hapoel players reached double digits in scoring in a 71-54 victory in Greece.
Olympia scored the first nine points of the game but Hapoel found their rhythm and ended up taking a 20-17 lead by the end of the first quarter.
The contest remained tight for the entire second quarter with Moran Roth's jump shot giving the Israeli team a 33-32 half-time advantage.
Despite the offensive fireworks of Michael King, who hit six three-pointers and had a game-high 25, Hapoel took control.
Leading 41-40 after two Sven Schultze free throws for Olympia, Hapoel outscored the Greek outfit 14-4 over the last 5:43 of the third quarter to lead 55-44 and stayed on top the rest of the way.
Hapoel are now 1-1 in Group E while Olympia are 0-2.
Guy Goodes (Hapoel Jerusalem coach):
"It was important for us to win today, after losing at home to Amsterdam on day one. We knew that Olympia Larissa is a good and aggressive team so we prepared to face that. Our target has been to reach the finish first or second to go through to the next round, and winning today cancelled the previous loss and renew our hopes. We were bad in the first half today but in the second we played better and managed to win the game so we are pleased."
Giorgos Bartzokas (Olympia Larissa coach):
"It was the worst game of the season for us and I consider myself responsible for not preparing my team properly for this game. I don't know if we can make it, playing in two different tournaments, EuroChallenge and the Greek league at the same time. We have two games every week, that makes us tired and today we had no motivation to win the game. I have to congratulate Hapoel and their coach for winning."
Erez Markovich (Hapoel Jerusalem):
"This is a very important win for us. We have to look forward to the next game, which can be our passport for the next round."
Marios Batis (Olympia Larissa):
"When you lose a game, it is never one man's fault. We are all equally responsible."