Mfcttrf

Historically, one reason is that the needs of "modern" banking (that which emerged during the 18th century) more-or-less required that banks were close to each other, since much of their business with each other – in particular the settling of payments by cheque – was conducted "in person". Quoting from the section 17th–19th centuries – The emergence of modern banking on Wikipedia's History of banking page (my emphasis):

The modern bank

In the 18th century, services offered by banks increased. Clearing facilities, security investments, cheques and overdraft protections were introduced. Cheques were invented in the 1600s in England and banks settled payments by direct courier to the issuing bank. Around 1770, they began meeting in a central location, and by the 1800s a dedicated space was established, known as a bankers' clearing house. The London clearing house used a method where each bank paid cash to and then was paid cash by an inspector at the end of each day. The first overdraft facility was set up in 1728 by the Royal Bank of Scotland.

As bank grew, and spread throughout the country (or acquired the provincial banks already there), the process of cheque clearing expanded: many cheques would be cleared locally (many/most cheques drawn on banks in, say, Bristol, would be presented to customers of other banks in Bristol), but those that weren't would make their way back to banks' head offices in the City of London where they would be exchanged at the Bankers' Clearing House on Lombard Street.

Similarly, for most of its history (up until the so-called "Big Bang" of 1986), trading on the London Stock Exchange was done in-person on the floor of the exchange. It therefore made sense for the companies involved to be located nearby.

During the late 19th, and early 20th centuries, as banking became more international, it would be natural for foreign banks to also congregate in the City of London, as that was where the exchanges and other banks were: they would want proximity both when negotiating deals and for day-to-day settlement. (While at university in the early eighties, I had summer jobs with what was then Morgan Guaranty Trust. They, like other banks, had an army of couriers who went on several "runs" throughout the day delivering cheques, shares and other legal documents to other institutions in the "square mile").

With the advent of electronic trading (on the stock exchange) and the rise in electronic payments (BACS – Bankers' Automated Clearing Services; similar to ACH in the United States), there has been a shift towards digital proximity being more important, which – together with more modern facilities – is probably behind the rise of Canary Wharf and the surrounding Docklands as London's second financial centre.

The main advantage to the City of London is that all the big banks are there. So there’s a large pool of experienced employees, there are legal firms with vast experience in the banking industry, there are accountancy firms with vast experience in the banking industry, there are secure courier companies and good telecommunications and temp agencies with lots of banking staff, and in general everything is set up to service banks. If you try to put your international bank’s European head office in Birmingham, you will have none of these things available.

Quite often competitors co-locate in order to increase business. Some business owners, see as it negative when a competitor moves in "across the street", but some see it as a positive.

An example of this is the diamond district in NYC. People come from all over the world to buy and sell their diamonds. They do this because of selection, and the convenience of finding an alternative if their first or fifth attempt at a transaction does not work out.

Another is in my town. The wife and I were in the market for some new furniture. We went to "furniture row", about 20 furniture stores within 3 miles. The brand we selected could have been purchased within a few minutes drive from our home, but instead we drove across town so we could shop around.

In a similar way, many world wide cities have "banking", or "garment" districts. Grouping together help serve their customers better, increases bushiness, and they can also work together on certain projects.

It is impossible to say why London specifically has a international banking district, but this pattern is repeated world wide for many different industries.

To supplement the present answers: one important reason was to ensure that Britain's financial leaders could be summoned quickly during a crisis. That changed when the market was deregulated by Margret Thatcher.

Until then the Bank of England had insisted that all of London's banks
had to be within 10 minutes' walking distance of the governor's office
so, it was said, in a crisis he could summon the lords of finance to
his parlour with half an hour's notice.

In fact, City of London is a Tax haven as point out by article like this. It is administered by City of London Corporation and not under the jurisdiction of Greater London.

This explains why even with a property price tag of £17,371 per m², there are many companies willing to pay the premium rent to open a business there. And you can bet even with all those "cost-cutting practices", those bankers are not going to move out of the City of London to save the rental.

p/s: Tax haven has nothing to do with how Bank paying tax. Banks are taking a cut by "helping" their client by "saving tax". One can read how the operation inside City of London is saving companies tax.

This is not only typical for banks in London, but to other types of businesses everywhere. Similar businesses tend to stick close together, an interesting phenomenon explained in Why do competitors open their stores next to one another?, an animated lecture by Jac de Haan.

Basically, if there is a physical distance between you and your competitor, then customers located between you and your competitor will more likely be going to you if you moved closer. If you moved further away, you would lose those customers.

Yes, unlike ice cream stands on a beach selling similar ice cream, the choice of a bank does not solely depend on how far it is from you, but it might play at least some role.