He left to found his own company, LoveFrom, which has worked with companies such as Airbnb and Moncler.
In the announcement about the merger, it said LoveFrom had been “quietly collaborating” with OpenAI for two years.
The idea for io, which Sir Jony founded last year with others, followed this partnership.
“It became clear that our ambitions to develop, engineer and manufacture a new family of products demanded an entirely new company,” said Sir Jony and Mr Altman.
OpenAI had a 23% stake in the start-up prior to Wednesday’s announcement, according to US media.
Ben Wood, chief analyst at CCS Insight, told BBC News that it would be “foolish to bet against Jony Ive, given his remarkable track record of delivering products that disrupt a market”.
“There is no question that OpenAI would love to have a direct relationship with its customers rather than delivering services via devices made by or powered by Apple, Google, or others,” he added.
Justin McGuirk, from the UK’s Design Museum, said Sir Jony “brings serious design credibility”.
He said: “If OpenAI want to take AI-based hardware to market, especially at a time when many are sceptical about the need for such things, it’s going to have to be incredibly good.
“Ive’s name will boost confidence and keep the hype machine rolling.”
