In ancient times, the city of Neapolis, dominated by Ephesus, was located on the mainland opposite the island. The heyday of this city began in late antiquity with the decline of Ephesus and the early Middle Ages through the increasing silting up of the port there.
Until the 15th century, the city was a commercial centre of the Republic of Venice and the Republic of Genoa under the name of Scala Nova.
In 1413 the Ottomans under Sultan Mehmed I conquered the city, which from then on belonged to the Ottoman Empire.

Already the Byzantines had established a first fortification on the island to protect the city.
The inner castle walls were commissioned by the Ottoman admiral Barbaros Hayrettin Pasha (ca. 1478-1546) and erected at İlyas Agha.

The outer walls were built in the second half of the 18th century to protect the island from attack during the Greek Orlov uprising (Greek uprising against Ottoman rule in 1770, named after Count Orlov, officers of the Russian army).

A 3 m high castle wall completely surrounds the island. The castle gate is located south of the castle and is flanked by two towers. The northern tower is pentagonal, while the southern tower is cylindrical. While an inscription on the northern tower dates from 1826, the inscription above the castle gate was lost. An empty space, today filled with a modern inscription plate, indicates its former existence.