What are the Session State Modes/ Session States

Storage location
InProc - session kept as live objects in web server (aspnet_wp.exe)
StateServer - session serialized and stored in memory in a separate process aspnet_state.exe). State Server can run on another machine
SQLServer - session serialized and stored in SQL server

Performance
InProc - Fastest, but the more session data, the more memory is consumed on the web server, and that can affect performance.
StateServer - When storing data of basic types (e.g. string, integer, etc), in one test environment it's 15% slower than InProc. However, the cost of serialization/deserialization can affect performance if you're storing lots of objects. You have to do performance testing for your own scenario.
SQLServer - When storing data of basic types (e.g. string, integer, etc), in one test environment it's 25% slower than InProc. Same warning about serialization as in StateServer.

Performance tips for Out-of-Proc (OOP) modes
If you're using OOP modes (State Server or SQL Server), one of your major cost is the serialization/deserialization of objects in your session state. ASP.NET performs the serialization/deserialization of certain "basic" types using an optimized internal method. ("Basic" types include numeric types of all sizes (e.g. Int, Byte, Decimal, String, DateTime, TimeSpan, Guid, IntPtr and UIntPtr, etc) .
If you have a session variable (e.g. an ArrayList object) that is not one of the "basic" types, ASP.NET will serialize/deserialize it using the BinaryFormatter, which is relatively slower.

So for performance sake it is better to store all session state data using one of the "basic" types listed above. For example, if you want to store two things, Name and Address, in session state, you can either (a) store them using two String session variables, or (b) create a class with two String members, and store that class object in a session variable. Performance wise, you should go with option (a).

Robustness
InProc - Session state will be lost if the worker process (aspnet_wp.exe) recycles, or if the appdomain restarts. It's because session state is stored in the memory space of an appdomain. The restart can be caused by the modification of certain config files such as web.config and machine.config, or any change in the \bin directory (such as new DLL after you've recompiled the application using VS) For details, see KB324772. In v1, there is also a bug that will cause worker process to restart. It's fixed in SP2 and in v1.1. See KB321792.
If you're using IIS 6.0, you may want to go to IIS Manager, go to Application Pools/DefaultAppPool, and see if any of the parameters on the Recycling and Performance tabs are causing the IIS worker process (w3svc.exe) to shutdown.
StateServer - Solve the session state loss problem in InProc mode. Allows a webfarm to store session on a central server. Single point of failure at the State Server.
SQLServer - Similar to StateServer. Moreover, session state data can survive a SQL server restart, and you can also take advantage of SQL server failover cluster, after you've followed instructions in KB 311209.

Caveats
InProc - It won't work in web garden mode, because in that mode multiple aspnet_wp.exe will be running on the same machine. Switch to StateServer or SQLServer when using web garden. Also Session_End event is supported only in InProc mode.
StateServer - In a web farm, make sure you have the same in all your web servers. See KB 313091 on how to do it. - Also, make sure your objects are serializable. See KB 312112 for details. - For session state to be maintained across different web servers in the web farm, the Application Path of the website (For example \LM\W3SVC\2) in the IIS Metabase should be identical (case sensitive) in all the web servers in the web farm. See KB 325056 for details
SQLServer - In v1, there is a bug so that iif you specify integrated security in the connection string (e.g. "trusted_connection=true", or "integrated security=sspi"), it won't work if you also turn on impersonation in asp.net. This problem is (wrongly) described in KB 324479. Unfortunately, the "Description" and the "Cause" sections in the KB are quite wrong and misleading. But anyway, there is a QFE fix for it, and the fix will also be available when SP3 ships. The problem is fixed in v1.1. - Also, make sure your objects are serializable. Otherwise, your request will hang! See KB 312112 for details. The SQLServer mode hanging problem was fixed in v1.1. The QFE fix for KB 324479 also contains the fix for this problem. The problem will be fixed in v1 SP3 too. - For session state to be maintained across different web servers in the web farm, the Application Path of the website (For example \LM\W3SVC\2) in the IIS Metabase should be identical (case sensitive) in all the web servers in the web farm.