In the spirit of Bil Simser's advice for SharePoint newbies, here's my half-cent's worth of contributions for those just starting down our treacherous path:
A common mistake that I see organizations make with large portal implementations is succumbing to the mysteriously irresistible urge to add all sorts of custom, third-party and external applications to their SharePoint server farm. Perhaps because they have invested heavily in the hardware to scale their portal and want to wring every last drop of processing power from the cluster, or they have a desire to maintain some sort of URL continuity (like decreeing that all applications from this point forward will start with http://somecompanyportal/ whether they need to or not), or maybe because SharePoint exerts some sort of Bermuda Triangle effect on administrators, attracting web applications like giant moths to the flame, within a month of a new installation I invariably discover all sorts of strange things going bump in the front-end web servers.
While there is no set rule on what should and should not be added to a SharePoint server - and SharePoint certainly has the ability to live side-by-side with all other types of creatures - common sense should dictate that just because you can heap on every web app the company has ever developed into a tangled maze of unmanaged paths, that doesn't mean you should. There are a few important things to consider before you go installing that nifty new copy of Crystal Reports Server on your FE's:
1. ISAPI Filters
SharePoint works a lot of it's magic through a custom ISAPI filter called STSFLTR. This filter handles all incoming and outgoing traffic through IIS, passing requests to the proper DLL's and sending back customized responses (like alternate URL's). In order to redirect traffic around STSFLTR you have to create an unmanaged path; that is, a virtual directory that is not part of SharePoint. When STSFLT intercepts traffic to/from any virtual directory, it first checks to see if that path has been marked as excluded. If so, it stops processing and hands the traffic off; if not, it handles all traffic for that path. As you can imagine, the more unmanaged paths, the harder STSFLTR has to work. Since it already does a tremendous amount of heavy lifting, the more overhead you add vis-a-vis managed paths the greater the impact on performance.
As if unmanaged paths aren't bad enough, many applications install their own ISAPI filters. Depending upon what order these filters are in on any given virtual server, you may experience anything from the odd quirky page to complete portal failure. ISAPI filters exist for the very purpose of managing traffic before it gets to the web application; the more hurdles your traffic has to clear before getting to its destination, the greater the odds that something will get fall down along the way. While third-party ISAPI filters may or may not be a supported configuration, I can assure you that it is not a recommended configuration. I have yet to meet an ISAPI filter that has not wreaked havoc upon the portal at some point or another.
Bottom line - avoid adding any ISAPI filters to a virtual server running SPS or WSS. Capice?
2. Hardware Resources
Ever watched the performance counters on a medium server farm with 3000 users? All those people uploading files, making web-part connections, polling the virtual servers for web part galleries, querying web services for data view web parts, and the umpteen gazillion database calls make the trace grid look like a politician's polygraph in the last week of an election. Now image what might happen when you bolt on that payroll app that uses an ODBC client to suck every last byte out of the Peoplesoft Oracle database. It ain't pretty when all that CPU crunching brings the virtual server to its knees. About the time it starts taking users 20 minutes to add a content editor web part to their personal site is when your Blackberry starts smoking like a Rastafarian bus driver.
Network traffic is another hidden minefield for any portal administrator. Chatty programs are the death of web applications; all those bits running amok makes it hard for the good stuff to get through. Remember that pre-SP2 most SharePoint implementations used host headers, adding overhead to every web request handled by the FE's. Be wary of any applications which make too many TCP-based database calls or custom ports, which are often a good sign that a lot of yapping is about to occur. Http traffic is http traffic, and the more of it the NIC has to handle the less bandwidth available for those big portal pages (have you ever looked at the total size of a standard web part page when all the linked content has been rendered? Here's a hint - OWS.JS alone is nearly 400k. You do the math). Sure, you can put in gig switches and prioritize traffic and add NICs and so on but I've got a better idea: how about you just put that stuff on a different server and save yourself the headaches?
3. Service Levels
It never fails - users start screaming that the portal is down and everyone points their finger at the SharePoint admins. Like that data view web part you just added really caused all the WSS sites to implode like the outer wall during the battle of Helm's Deep. Please. While we may shoot ourselves in the foot from time to time as we try to figure out how to do something simple in the ever-so-elegantly-designed SPS admin pages (That's sarcasm, in case you missed it. Hello, Redmond, are you listening?), eight times out of ten some rinky-dink app has caused the worker processes to lock up and slammed the door on our highly-scalable, disaster-resistant, backed-up and failed-over portal cluster.
Is it really a good idea to jeopardize your hard-earned five nines with that ever-so-important holiday party RSVP app? If you absolutely have to know if Suzy Smartypants is bringing her husband and thirteen children to the summer picnic, is it too much to ask for the developers to write a web part instead of redirecting to a clunky ASP page? Last time I checked, Web Part Library is on the same page as ASP.NET Application under New Project in Visual Studio. Newsflash: SharePoint is an enabling platform; that means you can use it to D-E-V-E-L-O-P W-E-B A-P-P-S. Now quit asking us to add another virtual directory for your latest project "just this once" and learn what Protected Override Void RenderWebPart(HtmlTextWriter output) does. Got it?
Now, go forth and erect thy fortresses of SharePoint, letting no man (or wayward application) penetrate thy defenses...
UPDATE: After reading this post, a colleague pointed out that in some cases the business justification for a high-performance cluster includes the value of running other applications alongside SharePoint (as in "We need this hardware but we can't get it without giving some CPU cycles to the Finance app:). Okay, that's a fair rebuttal - it's harder than ever to get management to sign off on big capital expenditures so you do what you have to do to get what you need.
In such cases the argument shifts from how the SharePoint environment SHOULD be constructed to how much RISK you are willing to assume based on business needs. I don't have any solid metrics to apply here, but it should be fairly easy to come up with a quick value matrix that weights such factors as performance degradation, anticipated downtime, CPU cycles, disk cost, etc. and score each projected application using these factors. This would provide you with a rough idea of the impact each addition would have on your SharePoint environment. You can then respond to other stakeholders with that information and, depending upon the relative importance of each application, let the powers that be determine what level of risk they are willing to assume.
If you follow the guidelines mentioned here when designing your matrix and ensure that new applications do not violate any of these key factors (forcing all other apps to use their own virtual servers and application pools, for example, or adding bandwidth for chatty programs such as web services) you can at least minimize the impact of living with troublesome neighbors.
