Since the infection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in several somatic cells, little is known about the infection of SASRS-CoV-2 and its related pangolin coronavirus (GX_P2V). Here we present for the first time that SARS-CoV-2 pseudovirus and GX_P2V could infect lung progenitor and even anterior foregut endoderm cells causing these cells death, which differentiated from human embryonic stem cells (hESCs). The infection and replication of SARS-CoV-2 and GX_P2V were inhibited when treated with whey protein of breastmilk and Remdesivir, confirming that these two viruses could infect lung progenitor and even anterior foregut endoderm. Moreover, we found that SARS-CoV-2 pseudovirus could infect endoderm and ectoderm. We found that whey protein blocked SARS-CoV-2 infecting these cells. In line with the SARS-CoV-2 results, GX_P2V could also infected endoderm and ectoderm, and also was inhibited by Remdesivir treatment. Although expressing coronavirus related receptor such as ACE2 and TMPRSS2, mesoderm cells are not permissive for SARS-CoV-2 and GX_P2V infection, which needed further to study the mechanisms. Interestingly, we also found that hESCs, which also express ACE2 and TMPRSS2 markers, are permissive for GX_P2V but not SARS-CoV-2 pseudovirus infection and replication, indicating the widespread cell types for GX_P2V infection. Heparin treatment blocked efficiently viral infection. These results provided insight that these stem cells maybe provided a stable repository of coronavirus function or genome. The potential consequence of SARS-CoV-2 and animal coronavirus such as GX_P2V infection in hESCs, germ layer and induced progenitors should be closely monitored.