Lessons learned from trying several e-mail providers
First and above all else, make sure you own the domain name. Switching e-mail address is tiresome and you cannot know whether your chosen provider will still be around in 10 years (or whether you would still wish to be their client by then).
Second, end-to-end encryption for e-mail means at best that your e-mails are encrypted when they sit in your inbox. Unless you are extremely picky with who you send e-mails to (or receive e-mails from), you should assume that all e-mails sit unencrypted in the other party's inbox. Whilst your own encrypted inbox is better than nothing, there are unfortunately significant trade-offs: your e-mail will typically only work with the e-mail client of the provider and your search experience will be worse. If you are sending sensitive information, it is highly recommended to use dedicated tools which are encrypted by default for both sender and receiver.
Third, more features typically means a higher price. To give you some perspective, a solid e-mail provider for your own domain starts from USD 10 per year. If you can live with using third party e-mail clients (such as Thunderbird or Outlook) and the occasional Roundcube (webmail), you can save yourself a lot of money.
Last, be wary of package deals. Centralizing all your services with one provider makes you dependent and likely harms competition in the long run. Whilst it makes sense to combine e-mail, contacts and calendar, ask yourself whether you need the other services in the package.