Chris Finch's team won 82-76 to improve to 3-1 in Group F while Khimik slipped to 2-2.
Elton Nesbitt poured in 19 points and Caleb Green 18 for Dexia.
Khimik were on top 70-69 with 5:17 to go after a jumper from Sasa Bratic.
Adam Hall answered with a three-point play for Dexia and then sparked the team with his defense, coming up with a steal and going in for a dunk for a 74-70 advantage.
Green then made two free throws and nailed a jumper to cap the 9-0 run and give the hosts a 78-70 lead.
Dexia held on for a six-point triumph.
Bratic had 19 points for Khimik.