Add Simon Cowell to the list of people not thrilled with the rumor that Howard Stern could fill his spot on 'American Idol.'

Stern doesn't have the experience to sit in his seat, the exiting judge Cowell said yesterday.

"You've just got to know what you're talking about," Cowell said, according to the Daily News. "We have to have people on these shows who actually know what they're talking about, rather than just guessing. They've really got to have experience, so that they can not just criticize, but offer constructive advice as well."

Cowell was "fairly certain" Stern hasn't been approached by 'Idol' producers to fill the spot when he leaves the show to host his own 'X Factor' show. The Stern 'Idol' rumor has run rampant on the Internet and has been partially fanned by the radio show host himself.