Cal takes a bite of pizza to give himself a few seconds to recover.

After he swallows he says,

"Yeah, but it's not my party, Mother'll invite whoever makes us look good. He won't be there, though. Uh - no offense, but nobody cares about Stark Industries right now. I mean, nobody we need to care about."

That and after the events in New York, Violet wouldn't dream of inviting Obadiah anywhere near her son, regardless of social relevance.