Boris Johnson's chief adviser, Dominic Cummings, has attended meetings of the scientific body advising the government on the coronavirus outbreak.
However, Downing Street denied a report in the Guardian that Mr Cummings was a member of the Scientific Advisory Group for Emergencies (Sage).
The committee, whose role is to provide independent advice, has helped shape the UK's response to the pandemic.
Labour said Mr Cummings' attendance raised "significant questions".
The Guardian reported that Mr Cummings and a data scientist who worked with him on the Vote Leave campaign during the Brexit referendum, Ben Warner, were among 23 people at a Sage meeting on 23 March.
That was the same day the prime minister announced the nationwide lockdown, bringing in strict new measures to tackle coronavirus.
A No 10 spokesman said Mr Cummings and Mr Warner had attended or listened in to Sage meetings in order to better understand the scientific debate around coronavirus.
He said they "occasionally" asked questions or offered help when "scientists mention problems in Whitehall", adding that others "also listen to meetings without being on, or a member of, Sage".
"The scientists on Sage are among the most eminent in their fields," the spokesman said. "It is factually wrong and damaging to sensible public debate to imply their advice is affected by government advisers listening to discussions."
He added: "'Public confidence in the media has collapsed during this emergency partly because of ludicrous stories such as this."
Sage is a panel of medical and scientific experts, chaired by the UK's chief scientific officer Sir Patrick Vallance, that provides advice to the government during a crisis.
The make-up of the committee has been kept secret - although individual members can disclose they are part of the group - and its advice has been given to ministers but has not been made public.
'Compromises independence'
Shadow health secretary Jonathan Ashworth said Mr Cummings had "no place" on the government's scientific advisory group.
"He is a political adviser, not a medical or scientific expert. If the public are to have confidence in the Sage, the government must make clear Dominic Cummings can no longer participate or attend."
Sir David King, the government's former chief scientific adviser, said he was concerned by Mr Cummings' attendance because if he participated in the discussion "this must compromise the independence" of the group.
He told BBC Newsnight: "What we don't know is the influence he plays in taking what he interprets from those meetings back to the prime minister."
BBC Newsnight policy editor Lewis Goodall said the the disclosure about Mr Cummings would contribute "to further calls for Sage and the advice that it gives to the government to be published and its membership to be fully disclosed".
On Friday, the UK's chief medical officer Prof Chris Whitty, a member of the body, told MPs he had no objection "in principle" to details of the membership being released.