Sources connected to the production tell us ... they were in a mad scramble when they were contacted 4 days before Saturday's taping and informed the Hiltons weren't playing.
Our sources say Paris and Kathy were on-board, but patriarch Rick Hilton was having none of it. We're told Rick -- who runs the monster real estate company Hilton & Hyland -- felt it wasn't right for him to appear because he's not in entertainment ... and it looked "thirsty."
We're told Nicky thought it was nuts for the family to appear so she was out, and Barron and his fiancee ultimately decided not to do it.
Fact is ... the families go back 3 decades and they really like each other, but it wasn't right for the Hiltons.
So instead, the Kardashians/Jenners and the Wests battled it out ... Donner Party style.