At the time, he defused the situation by ordering them to do press-ups and joining in.
Mr Abiy and the soldiers were seen laughing but he told parliament that “inside I was very unhappy”.
Over the past six months, he has made some big reforms and he said some of the troops wanted to derail them.
“The march of some members of the army to the National Palace [the prime minister’s office] was not only unlawful but very dangerous, because the intention was to abort the ongoing reforms,” Mr Abiy told MPs during a question-and-answer session.
“Meanwhile, after the situation was brought under control, some forces were heard saying: ‘He escaped before we could kill him.'”
On 10 October, several hundred protesting soldiers, some of them armed, went to see the prime minister – reportedly to demand a pay rise.
The situation caused alarm, leading to road closures in the area and the internet to be shut off for hours.
Speaking to MPs on Thursday, Mr Abiy said that without the order to do the press-ups, “the whole issue would have escalated… Some people see the push-ups we did as very casual. But, we used it to pacify the situation.
“In the military, push-ups or doing some kind of exercise or talking loudly are some of the measures used to relieve grievances or emotions.”
Since coming to power in April, Mr Abiy has made some dramatic changes – including freeing thousands of political prisoners, unbanning some outlawed groups and making peace with long-time foe Eritrea.
Last month, Ethiopian prosecutors charged five suspects with terrorism over an attempt to kill Mr Abiy in a grenade attack at a rally in June.
He escaped uninjured and described the attack at the time as an “unsuccessful attempt by forces who do not want to see Ethiopia united”.