Pence did not directly address news of the invitation when asked.
The North Korean ferry Mangyongbong-92 arrived in the South carrying the performers on Tuesday and also served as their accommodation during their stay in Gangneung.
They said Pence could have opted to sit with the U.S. delegation and avoided the box that included the North Koreans.More news: Kim Jong-un invites South Korea's President to Pyongyang (Second Lead)
But he said it would be harder for Moon to keep up momentum after U.S. and Japanese allies and South Korean conservatives reminded him of what was at stake in the North Korean nuclear threat.
For months now, the world has been grappling with fears of a nuclear war as North Korea has steadfastly refused to toe the US's line on nuclear de-escalation leaving Pence with no option but to keep piling on sanctions on the country with one of the worst human rights record globally.
But both South Korea and the USA has been cautious about the talks, saying there will not be progress without North Korea giving up nuclear weapons.
The first few days of the XXIII Olympic Winter Games have already revealed the softer side of the North's effort to woo the South and soften the resolve of the US and its allies.
A White House official said that, although Moon did not discuss the invitation with Pence on Saturday, the South Korean president made it very clear that only when North Korea actually starts to take steps to de-nuclearise would anyone even consider beginning to take the pressure off. "A bunch of journalists were chasing us so security took us outside so we could get to our seats", the fake Kim added.More news: South Africa's ruling ANC to finalise talks on Zuma's future
The Trump impersonator was identified as American Dennis Alan, while the Kim Jong Un impersonator is from Australia.
This will also be challenging for the Moon administration given the continuing approach by the US, South Korea's key ally, to exert maximum pressure on North Korea to abandon its weapons programmes.
Moon should recognise that he himself has to press North Korea to denuclearise, rather than relegating the issue to dialogue between the United States and North Korea.
The two Koreas held summits in 2000 and 2007.
South Korea's Unification Ministry spokesman Baik Tae-hyun told a regular briefing Seoul was still in close consultations with the United States over its North Korea policy. US President Donald Trump and the North Korean leadership traded insults as tensions rose.More news: Olympic Results: Germany Leads Medal Count
- 'Frank, candid' talks held with Moon, says Pyongyang
- Antonio Conte reiterates desire to fulfil Chelsea contract
- Gigi Hadid slams body shamers
- Trump To Unveil $1.5 Trillion Infrastructure Plan Days After Signing Massive Budget
- NASA probe "New Horizons" snaps farthest-ever photo from Earth
- South Korea's Leader May Be on Brink of Legacy-Defining Moment
- Trump and Kim lookalikes cause a stir at Pyeongchang Olympics
- Sensex rebounds 295 pts on positive global cues
- Hawks fans taunt Cavs with epic LeBron chant
- Philippines Bars Citizens From Working in Kuwait, Citing Abuses