The die-hard Swedish arrived in Latvia with just one win in three games but made clear they would fight to the last man as they grabbed a 35-23 lead late in the first quarter.
Berzins and Janis Blums instigated a 13-2 Latvian burst at the start of the third quarter and the hosts claimed a 42-39 lead on a Martins Meiers jumper wit 4:52 remaining in the third quarter.
The two sides exchanged punches from then on until Tobias Borg, with two shots from the charity stripe 47 seconds from the end of regulation, gave Sweden a 64-60 cushion.
Janis Timma sunk two free-throws 37 seconds from the end of regulation to reduce Sweden's lead to 64-62 and on the other end of the floor, Berzins came up with a huge block, his fourth of the night, as Joakim Kjellbom was attempting to score.
Latvia were presented with the chance to tie the scores or push ahead but, with nine seconds left on the clock, Janis Strelnieks turned the ball over and had no chance but to foul Jonathan Person.
The Swedish guard went to the line and made his first free throw to increase the gap to 65-62 but missed the second.
Latvia got the rebound and Sweden fouled Strelnieks, with just three seconds left on the clock.
He made his first shot from the stripe and missed the second for Meiers to come up with the offensive board.
He missed the lay-up in traffic, but deus-ex-machina Berzins tipped the ball in, right at the buzzer, to tie the scores at 65 and send the game to overtime.
Latvia pushed ahead immediately in the extra period but Anton Gaddefors drilled a huge triple with four seconds left on the clock to reduce to 77-76.
The hosts called a time-out and once play resumed, Sweden immediately fouled Kristaps Janicenoks.
The experienced guard kept his calm at the line and made both his free-throws and Sweden did not manage to fire off a desperation shot in the two seconds that elapsed until the final buzzer.
Berzins finished with a double-double of 23 points and 10 rebounds while Strelnieks added 14 points, six boards and six assists for the winners.
For Sweden, Kjellbom collected a double-double of 19 points and 13 boards and Gaddefors added 15 points in losing efforts.