Michael Madison, 35, was arrested Friday after authorities uncovered the body of a woman who had been wrapped in several garbage bags in a garage near a vacant East Cleveland home where he was apprehended.
Madison, a convicted sex offender, has been named a suspect in all three deaths, Norton said. He has not been charged.
"We believe every hour he was on the street, he would have been a danger to someone," East Cleveland Mayor Gary Norton told ABCNews.com.
Police returned to the neighborhood to find two more bodies in the backyard and in the basement of nearby vacant homes the next day, Norton said.
All three corpses were found in the fetal position, wrapped in multiple trash bags, officials said.
While the identities of the three victims are currently unknown, Norton said that investigators believe all of the victims were black women who were killed within the last six to 10 days.
Based on investigators' initial reports, only one of the bodies was found without clothing. While Norton could only provide details on the shirts the other two women had on, he did not know if they were fully dressed when they were uncovered.
East Cleveland Police Chief Ralph Spotts called the discoveries an "absolute worst case scenario."
Norton said families who have relatives missing called in as soon as news of the discoveries broke.
"Some concerned family members can take comfort in knowing their loved one is not one of the victims," Norton said. With no autopsies scheduled at this time, others will have to continue to wait, he said.
The Cuyahoga County Medical Examiner's Office confirmed the three women were found "in states of advanced decomposition," making it difficult to determine their final causes of death, but were not soliciting DNA samples from families of missing persons at this time.
Despite the fact that Madison was registered as living at his mother's home several miles away from where the bodies were found, he was well-known in the neighborhood.
"He was a person who frequented the area, and had spoken to and approached multiple women in the area," Norton said.
It is unclear where Madison's mother is, he said, or where he was living at the time of his arrest.
Police and the FBI expanded their search today to more vacant homes and empty lots near the neighborhood where Madison frequented, after he led investigators to believe he may have been influenced by Ohio serial killer Anthony Sowell, Norton said.
Police will also be looking into anybody who was recently reported missing, the police chief said.
Sowell, who was described in court papers as "the worst offender in the history of Cuyahoga County and arguably the state of Ohio," was sentenced to death in 2011 after he was found guilty of killing 11 women and hiding their remains around his Cleveland home from June 2007 to July 2009.
"[Madison] said some things that led us to believe that in some way, shape, or form, Sowell might be an influence," Norton said. "Hopefully, we pray to God, this is it."
The gruesome discovery was made just 10 miles from the home where Ariel Castro allegedly held three women captive for nearly a decade until they were able to escape in May.
Despite the arrest of a suspect people in the East Cleveland neighborhood said they are still anxious.
"Something like this is very upsetting, not only for me but for my children and grandchildren," East Cleveland resident Bernadette Bass said.
Another resident said the discovery has convinced her it's time to find a new place to live.
"I have three daughters of my own and me and I do walk and go to the store, so it's kind of scary out here," the woman said. "I'm about ready to leave. Matter of fact I am moving."