Starr was cast as Bill Haverchuck in the NBC sitcom Freaks and Geeks.
After Freaks and Geeks, Starr had a number of guest appearances and one-time roles on shows such as Ed, Mysterious Ways, Providence, King of the Hill, and Normal, Ohio. He had a small recurring role during the third season of Roswell as the character Monk. In 2002, Starr appeared in the films Stealing Harvard and Cheats. He also reunited with both Freaks and Geeks producer Judd Apatow and Freaks and Geeks co-star Seth Rogen when he made a guest appearance on Apatow's situational comedy Undeclared in the episode "The Perfect Date". In 2005, he appeared in the miniseries Revelations and the CBS sitcom How I Met Your Mother.
Starr co-starred in the 2007 comedy film Knocked Up, which was directed by Judd Apatow. The same year, he had small appearances in Superbad and Walk Hard: The Dewey Cox Story, both of which were produced by Apatow. The success of Knocked Up led to Starr having larger roles in films, such as Good Dick and Adventureland. Starr also appears in many well received short films that premiere at film festivals, such as the Sundance Film Festival.
From 2009 to 2010, Starr was a series regular on the Starz sitcom, Party Down, portraying Roman DeBeers. Since 2011, he has co-starred on the Adult Swim television series NTSF:SD:SUV::, in which he plays Sam Stern. In 2011, Starr had a major role in the film A Good Old Fashioned Orgy and also had guest appearances on television series such as Mad Love, Community and Childrens Hospital. In the latter he reprised his role of Roman DeBeers from Party Down.
Starr reunited with Freaks and Geeks and Party Down cast member Lizzy Caplan in the 2012 film Save the Date, which premiered at the Sundance Film Festival. The same year he had a supporting role in the thriller film Deep Dark Canyon, as well as guest starring on sitcoms Parks and Recreation and New Girl. In 2013, he had a small role playing himself in This Is the End, which was directed by Seth Rogen and Evan Goldberg and had a starring role in the Kristen Bell film The Lifeguard, later working with her again in the film Veronica Mars. He also guest starred in The Aquabats! Super Show!, Franklin & Bash, and Drunk History. In 2014 he was casted to play Bertram Gilfoyle in the HBO comedy Silicon Valley. As of spring 2016, Starr is reprising his role as Gilfoyle in the third season of the show.