Klebsiella pneumoniae (Kp) is an important cause of healthcare-associated infections, which increases patient morbidity, mortality, and hospitalization costs. Gut colonization by Kp is consistently associated with subsequent Kp disease, and patients are predominantly infected with their colonizing strain. Traditionally, the indigenous gut microbiota is thought to protect individuals from colonization by bacteria with pathogenic potential, and disruption of protective gut bacterial communities permits colonization and subsequent infection. Interestingly, there are now several lines of evidence that this is not necessarily the case for Kp. Kp can survive in an unperturbed microbiome; however, the Kp factors permitting fitness in the intact gut microbiome are largely unknown. Our previous comparative genomics study, between disease-causing and asymptomatically-colonizing Kp isolates, identified a plasmid-encoded tellurite (TeO3-2)-resistance (ter) operon as strongly associated with infection. Using multiple mouse models of infection and colonization to show that 1) ter is dispensable during bacteremia, 2) ter enhances fitness in the gut, 3) this phenotype is dependent on the colony of origin of mice, and 4) antibiotic disruption of the gut microbiota eliminates the requirement for ter. Furthermore, using 16S rRNA gene sequencing, we show that the ter operon enhances Kp fitness in the gut in the presence of specific indigenous microbiota, including those predicted to produce short chain fatty acids. Finally, administration of exogenous short-chain fatty acids in our mouse model of colonization was sufficient to reduce the fitness of a ter mutant. Together, our data indicate a novel role for the ter operon as a microbiome-dependent gut fitness factor and that indigenous gut species may inhibit colonization by pathogenic Kp. Contextualizing these findings in a recent comparative genomics study focused on Kp infections with concordant colonizing strain, we demonstrate that there may be a broader role for plasmid-encoded factors during Kp infection and colonization. This work represents a substantial advancement in our molecular understanding of Kp pathogenesis and gut colonization, directly relevant to Kp disease in healthcare settings.