Wealthy art collectors Bruce and Robbi Toll had launched an appeal after a court ruled in November that the painting belonged by rights to the descendants of Simon Bauer, a Jewish businessman disappropriated by the Nazis in 1943.
The Tolls insisted they had no idea the painting, "La Cueillette" ("Picking Peas"), had been looted when they bought it at Christie's in New York in 1995 for $800,000 (692,000 euros).
But the Paris appeals court ruled Tuesday that the original court decision stood, in a move hailed by the Bauer family.
The ruling "gives victims of the savagery committed by the Vichy government the right to recover their looted possessions, without a time limit", their lawyer Cedric Fischer said in a statement.
The Vichy regime, France's anti-Semitic wartime government which collaborated with the Nazis, seized 93 paintings from Bauer.
The wealthy businessman narrowly escaped death when a train drivers' strike stopped him from being sent to a concentration camp.
Bauer recovered a few of his paintings after the war, but never La Cueillette, which Pissarro had painted in gouache in 1887.
- 'Bought in good faith' -
The verdict paves the way for the Bauer family to retrieve the painting, which during the court case has been kept locked up by the Musee d'Orsay and Orangerie museums.
The ruling mirrors other legal disputes over art and property looted from Jewish owners by the Nazis which were subsequently sold on to often unsuspecting new owners.
Out of 650,000 pieces of art stolen by the Nazis, about 100,000 had not been returned by 2009, according to figures released at the Holocaust Era Assets Conference held in the Czech Republic that year.
Bauer died in 1947, two years after the end of World War II.
His family spotted an opportunity to get the Pissarro painting back when it went on display at the Marmottan museum in Paris last year during a retrospective of the artist's work.
They argued in court that the Tolls, experienced collectors who made their fortune in real estate, must have known the painting was on a list of looted artworks.
But the court accepted the argument that the Tolls, who are patrons of the Washington and Tel Aviv Holocaust museums, bought it in good faith.
"It is not Mr Toll, who bought this painting at public auction in 1995, who should pay for the crimes of Vichy," their lawyer Ron Soffer had told AFP in November.
A government decree announced Tuesday that France would boost the powers of the commission which awards compensation to victims of Nazi looting.
The move, promised by Prime Minister Edouard Philippe over the summer, means the Commission for Compensating Looting Victims (CIVS) will now be able to launch an investigation on an individual's request and recommend appropriate compensation.
This includes artworks that have made their way into public collections or national museums, according to the decree published in the national gazette.
In 2011 a raid on a rubbish-strewn flat in Munich uncovered hundreds of priceless paintings, including works by Picasso and Matisse, that had been stolen by the Nazis.
The flat belonged to Cornelius Gurlitt, an octogenarian whose father was one of four art dealers tasked by the Nazis with selling the art.
An additional 239 works were found at a house he owned in Salzburg, Austria.