I'm not going to make a statement that understanding self-issued certificates is hard, but I do get enough questions on them that it's prudent for me to do a post regarding them to help folks understand how they work.
First, what's a Certificate used for?
An SSL Certificate is used for two reasons, to validate the remote server to the client, before the client sends any data to that server, and to encrypt the data between the client and server over an un-secure network (ie. the Internet). The former prevents DNS attacks such as Man in the Middle Attacks, or Phishing, by having the server present you with some valid Identification that it is who the server says it is. Think of it like Identification for you at a party or club. Let's say you work for a company called Contoso, they have photo id at the company which you have, and you also have a valid drivers license for the state you're in. Let's say you go to an invite-only party. If you show up at that door, you can show your drivers license to prove that you are who you say you are, this has your picture on it that the door man can see, and your name. The door man can look at the picture, and at you, and because he trusts the state government, knows that you are you. Think of the state-issued drivers license as a Trusted Certificate, because the door man trusts the issuer (or the state government).
Now let's say you show up at the same party, but this time the party is a party for Contoso. You could show your drivers license because the door man will trust the state-issued ID, or you could show your company ID badge, because the door man works for this company and trusts the badges issued by Contoso. Think of the company ID badge as a Self-Issued Certificate. Because the badge and ID was maintained by the company, not all locations will trust it, just the ones that trust that Issuing Authority (or the company Contoso's badges).
Back in the land of computers, let's take a look at what happens when a client connects to a secure server. To the left is the process that clients and server goes through. The client will resolve the server's IP address via DNS and end up at a server's web-site. That client will want to verify the server's identity, so it will request the server's certificate, or Identification. The server will provide this certificate to the client.
The client then looks at the certificate and validates 3 key things:
1) The date range for validity of the certificate. Is the current date on the client within the start/end dates for the certificate provided by the server?
2) The URL the client was attempting to go to is the URL of the certificate (e.g. www.contoso.com is both the URL that the client wants, and what the certificate was provided)
3) The Certificate is from a trusted provider. The client will look at the certificate chain and validate that the root certificate is located in the client's local trusted certificate store as a valid trusted certificate.
If all three of these checks pass, the client will then agree to trust the server and use the certificate to encrypt traffic when talking to the server, and request the web page the client was originally looking for. Please note here, that the client doesn't get to the web-site (including any IIS headers) until the certificate is validated. So if the server is hosting multiple web-sites, it has no idea which one the client is asking for until the process of validating the certificate is complete.
Once the certificates first purpose is completed (to ensure the client is actually talking to the correct server on the Internet), the second purpose kicks in and the client uses the public encryption key to send and receive information to and from the server. The server is the only machine on the whole Internet (if it's not, the root certificate needs to be recreated, unless there is trust between those multiple computers, as in a cluster.) that has the private key and can decrypt the network traffic.
You may wonder how the server identifies the client computer through all this? That's the easy part, the user logs into the web-site with a username and password, and sometimes additional authentication keys using 2-factor authentication.
So a Trusted Certificate is more secure right?
Wrong! A Self-Issued certificate, and a Trusted Certificate often provide the same level of encryption. The big difference between a self-issued and trusted certificate is the ease of use. Many trusted certificates are good for 1 year, and have 1024 bit encryption. The SBS self-issued certs are good for a period of time (5 or 2 years as you'll see below), and also have 1024 bit encryption. Where they differ is distributing the root certificate. In order for a client (PC, Windows Mobile, MAC, etc) to trust that certificate it must know about the root cert, which means in a self-issued scenario, that this cert must be distributed to all clients that are going to connect to the server. In a trusted certificate scenario, the trusted root certificate is already distributed by Windows Update, or at the time the PC, MAC or Windows Mobile device was built! So there is no need to distribute the root cert if you're using a trusted certificate, because the client already trusts the root cert.
So, your choice between Trusted and Self-Issued really comes down to how much work you want to do to distribute the root certificate, more so then the security level of the certificate. You need to consider all the devices that will connect. If one of those is a kiosk at an airport or hotel, in which you cannot install the root cert, then there is a clear choice for you towards Trusted Certificates. If you have remote workers, it's probably easier to get a trusted certificate as well.
(As a side note, 1024 bit encryption was chosen primarily because the processors in Windows Mobile devices get bogged down encrypting and decrypting traffic using higher numbers. I'm sure this will change over time)
Windows Small Business Server 2003 Self-Issued Certificate
In SBS 2003, the root certificate was also the leaf certificate. This means that the root certificate was the certificate used to validate trust, and also encrypt the traffic on the web.
As you can see, the certificate is the root certificate. This certificate needs to be sent to all the remotely connecting devices. If this certificate is not in the remote computer/device trusted authority certificate store, then the user will always receive a certificate warning that the certificate is not trusted. This root certificate is valid for 5 years.
Windows Small Business Server 2008 Self-Issued Certificate
In SBS 2008, we went a step further with the self-issued certificate, and provided a root certificate and a leaf certificate. The root certificate needs to be distributed to all the clients, and the leaf certificate is installed onto the IIS web site. This adds slightly more security over SBS 2003, because the leaf certificate is valid for 2 years, and the root certificate is valid for 5 years, similar to SBS 2003. The reason this is more secure, is because if the leaf certificate is compromised by any way, you can simply re-issue a new leaf certificate and not have to replace any of the certificates on the computer/devices. In fact, the Fix My Network wizard will do this for you when the certificate is about to expire.
You can see the chaining from the certificate path above. In addition to the new improved design of the certificate infrastructure, we also provided a ZIP file that can be used to distribute the package. This Zip file is available from "\\contoso-server\public\public downloads\" internally to the network only. Users will copy this ZIP package to a USB disk and take it with them to their home PC to install it.
Important: People always ask me why this isn't available directly from the Remote Web Workplace. Well, it goes back to the flow chart above. You can't download a certificate to trust a server from an un-trusted source! Essentially if you agree to go to the un-trusted source, you have no guarantee that the package you download is the right one. Not only that, but you will have already provided your username and password to an un-trusted server which could be the wrong one! If the un-trusted server was malicious, they could log into the actual server as the user and do all sorts of damage! Taking the package home on a USB key in your pocket is the safe way, because the certificate is traveling through a trusted path out to your remote PC.
In addition, because the Certificate Authority is configured and set up by default in SBS 2008, you can entertain the idea of using the certificate infrastructure to configure secure wireless, or IPSec, or secure VPN.
Recommendation
So, if you're still reading (long post today!), you understand how certificates work to secure network traffic and ensure you're communicating with the correct server. You understand the differences between the self-issued certificates, and trusted-authority certificates, and finally, how they are used in the two versions of Windows Small Business Server that use certificates.
If you wanted the Microsoft recommendation, for ease of configuration and explanation to your end users only, we recommend going with a trusted certificate for your network. Prices on trusted certificates have fallen to less then it would cost to pay a consultant a hour to install the remote devices, not to mention the education you need to provide your users. In speaking with many consultants, they'd rather spend their efforts elsewhere any ways! The self-issued certificate is really a means to provide encryption level security to businesses that absolutely cannot afford a trusted certificate. Trust me, it's worth it...
For bonus points, who can tell me why I call them self-issued instead of self-signed?
