Not to be outdone by Batman, Superman was also around during Sesame Street’s humble beginnings. Here is some info from Muppet Wiki:
Beginning in the first season, Superman appeared on Sesame Street in animated inserts, produced by Filmation, the studio which had produced The New Adventures of Superman (1966-1967). In his first appearance, as seen in the first test pilot, Superman battles a giant ape, in recycled footage from the New Adventures episode “The Chimp Who Made it Big,” and a rapid transition results in the Man of Steel briefly announcing the letter D. In the archive footage, Superman’s voice is that of Clayton Collyer, who played the role on radio and on the Filmation series. (Old School: Volume 2)
Superman’s second appearance featured a more substantial amount of new footage, and a new voice actor (Lennie Weinrib), as he explains why S is his favorite letter. (EKA: Episode 0184) (Old School: Volume 1)
Did you know in the early days of Sesame Street, Batman and Robin were part of the show? It’s true! Watch the clip above and read more about it from Muppet Wiki:
Batman’s association with Sesame Street predates the show’s premiere. In DC comics ads, alongside Superman, Batman urged young viewers to watch Sesame Street, especially “The Man from Alphabet” segments (which were dropped before the show’s national debut).
Batman appeared in several Sesame Street segments in animated form (voiced by Olan Soule). In an Ernie and Bert sketch from the first Sesame Streettest pilot, Ernie wants to watch Batman on TV, but Bert wants to watch something else. Batman interrupts them, suggesting they take turns in selecting a television program.
Batman was joined by Robin (Casey Kasem) in several animated segments (produced by Filmation) which debuted in the end of the first season. Gerald S. Lesser explained the selection of the dynamic duo when he said “some celebrities such as Batman and Robin are enlisted because of their unassailable authority with four-year olds.”