Along the same lines as this question from a players perspective, I am interested in possibly running a game for some friends and co-workers, where they have much more experience in gaming than I do in modern systems, especially mechanics based systems such as the proposed 5e campaign (Most likely one of the published adventure paths).

As experienced gamers, they are going to expect a certain level of knowledge and pace of play from the other people at the table, including the GM. However, until that base level of knowledge can be obtained, there will be rule interpretations, table/mechanic lookups, and similar that will slow down the pace, and basically be players sitting around waiting for the GM which is not really optimal for anyone.

What are the things that I can do beforehand and in game to mitigate the inexperience and keep things fun and interesting for the players?

Consider what campaign you want to run

There is a great video from Matthew Colville which explains this further but as a TL;DR think whether you want your players to follow a specific storyline (Railroad) or you want them to shape the storyline (Sandbox), it could even be both.

I would recommend somewhere in the middle, where you either buy a premade campaign or create your own with a certain story for example: Retrieve an ancient artefact that is located somewhere far away and is guarded by a very powerful creature but you allow your players to change the story as they go. They might feel that retrieving this ancient artefact is of no good use to them so you should give them a strong reason to set out on an adventure. Maybe their hometown is threatened by this creature and this so-called artefact is the only way to weaken it in order to kill it. (Something like a Tarrasque would be a challenge but I would recommend changing some stats to fit it in the setting of your campaign).

Learn as the campaign progresses

In the DMG (page 4) it says that as a referee the DM has to make sure all the rules are followed. As a new DM, I don't think that this should be your primary concern. Of course, you have to study the core rules of battles, stats and so on but I honestly believe that learning as you go is the best way to learn.

Since the players are experienced enough they might even help you in some situations. There have been countless times when I have forgotten some rule about a certain spell and a player reminds me.

Keep it consistent

One of the issues my party had on our first campaign (We're a bunch of friends playing together and at the time one of my friends was the DM) was that it wasn't consistent at all. We basically bought the Starter Kit and the thing is that the DM wanted to add many homebrew content which I believe wasn't the best option.

This homebrew content, though, made the campaign inconsistent. He basically gave us quests that he made up that were way out of where we were supposed to go. After completing one quest he then threw another "very important" one at us.

If you're going to add content to your campaign (And you should) make sure that it keeps the campaign consistent and it is important to remember that when the players have nothing to do they either stop playing or try to destroy what little there is. In our first campaign after the fourth or so, side-quest we decided to burn a village because; why not? We had nothing to do and it had started to get boring after five 6-hour sessions we had made no progress whatsoever towards our initial goal, a goal that was important to our characters.

Be descriptive

As a tabletop RPG, it is important to be as descriptive as possible. Setting the atmosphere is as important as describing the little details the players notice. For example, when the players walk into a tavern describe what they smell, what they see, what they hear.

In some cases, you might find it difficult to come up with a description of a very important place while you are preparing for your next session. At times you could use pictures and hand them out to your players so they can all feel like they are there.

Give your players freedom

Whether you want to run a Railroad or a Sandbox campaign it is important to allow your players to make decisions. They might feel limited at times but if it is important for the continuation of the campaign I am sure they won't mind it too much.

When I say freedom I mean from deciding to not retrieve the artefact to choosing in which inn they want to stay for the night.

Allow and reward roleplaying

Roleplaying is one of the most important aspects of an RPG. I highly recommend you allow your players to be able to roleplay, either between them or with NPCs. It is important for them to feel like they are a part of the world.

If a player roleplays a very difficult situation then reward them. If the party has been captured by the dark overlord of a country and a character tries to make a deal with the overlord then allow them to. This may change the entire course of the campaign but it may prove to be more interesting than what you had already planned.

Have fun

The most important thing is to have fun. If you don't know a rule, it's fine! You will learn it, if you can't come up with a description on the spot, that's fine as well I am sure they will understand. The main reason why you should be playing is not "to retrieve an artefact" but rather have fun.

Read as much as you can but learn as you go

This is what I did and it's turned out really fun! If you're with a good group of friends and they know you're not the most experienced they're not going to expect you to know everything! As long as you know the bones of the system the players themselves can help with the niche aspects of it.