IS targeted Shia soldiers from Camp Speicher, a former US Army base.
The June 2014 massacre became a symbol for IS brutality after videos and pictures were posted on social media.
Tikrit was retaken from IS control by the Iraqi army and Shia militias days ago after a month-long siege.
IS militants claimed at the time to have executed 1,700 Iraqi soldiers from Camp Speicher.
Work has started on eight locations inside Tikrit's complex of presidential palaces, where the killings are thought to have taken place, Kamil Amin, from Iraq's Human Rights Ministry, told the Associated Press.
The presidential compound of the former Iraqi president Saddam Hussein became the IS headquarters after they took the city last year.
"The work is continuing and we expect to discover more mass graves in different areas," Mr Amin said. "We expect huge number of bodies to be unearthed."
DNA testing will be used to identify the bodies once they have been exhumed, as many families have never had confirmation of the death of their relatives.
'Heartbreaking scene'"We dug up the first mass grave site today. Initial indications show indisputably that they were from the Speicher victims," Khalid al-Atbi, an Iraqi health official working with the forensic team, told the Reuters news agency on Monday.
"It was a heartbreaking scene. We couldn't prevent ourselves from breaking down in tears. What savage barbarian could kill 1,700 persons in cold blood?" he added.
The murder of the soldiers has become a lightning rod for Shia militias who have vowed to avenge the killings.
The militias have been credited with halting Islamic State's advance last year when they captured key cities and towns, but have themselves been accused of war crimes.
The offensive launched on 2 March to take back Tikrit involved some 30,000 fighters, two-thirds of them from the Popular Mobilisation (Hasid Shaabi) - a force comprising dozens of Iranian-backed Shia militia.
The Iraqi army is now expected to turn its attention to Mosul, 225km (335 miles) north along the Tigris river.
IS's most significant stronghold in Iraq, Mosul, presents a far greater challenge than Tikrit for the US-led air coalition and Iraqi ground forces.