I haven't introduced myself here before, but I've DM'd consistently for about 3 years now, and I figure I can help.

[hr]

[@Vertigo] You can literally get started for free if you're willing to put extra work in and cut certain corners, but money can save you time and effort.

You do not need the starter kit if you come up with your own campaign. This takes a lot of work, but is free, and I'd expect it to be more rewarding as a GM. It's all I've ever done, and Matthew Mercer, a DM for the stream Critical Role (and who is a very good example to follow as a DM - the streams are on YouTube), did the same. If you want to use a starter kit, like I said, I have no experience with them, but from what I've seen and heard they are good for inexperienced DMs. They will provide everything you need straight out of the box.

I should mention that the starter kits only provide rules for levels 1-5. If you want to go further than that, there are three choices I am aware of. The first is to use the [url=https://www.5esrd.com/]SRD[/url], which is completely free. It will provide rules for all levels, but will limit your options in terms of races, subclasses, monsters, and so on. The second is to play 3.5e instead. The rules for this edition are all available online as PDFs for free, just google the three core rulebooks: Player's Handbook, Dungeon Master's Guide, and Monster Manual. The rules for 3.5e are not as simple as 5e, but with these you will be able to play with all options in the game, and there is more supplemental content if you want that later. That is how I started. The last would be to buy the fifth edition rulebooks (Player's Handbook, Dungeon Master's Guide, Monster Manual). These are very expensive ($50 apiece if I remember correctly, but you can probably get them cheaper second-hand), but this way you would be playing with the complete game in the edition you wanted. That is what I currently do.

You don't need to buy dice, but I recommend it. You can get a full polyhedral set quite cheaply, or use an online random number generator if you prefer. I can't really explain why physical dice are better, I think it's just more fun to roll them.

Otherwise, I strongly recommend you watch Matthew Colville's series on helping out DMs - he is another great DM, and he recommends multiple pieces of software, books, and other tools in his series that make the development of a world and a campaign so much easier, if you decide to go down that route. I still use his advice now, and even if you decide to use a starter kit, it is extremely helpful.