book

Web Performance Tuning, 2nd Edition

by Patrick Killelea

March 2002

Intermediate to advanced

482 pages

16h 11m

English

O'Reilly Media, Inc.

Read now

Unlock full access

What Is This Book Good For?Audience for This BookAssumptions of This BookHow This Book Is OrganizedPart I: Preliminary ConsiderationsPart II: Tuning in DepthFont ConventionsHow to Contact UsWeb Site Updates and Code ExamplesOther Books and ResourcesBooksWeb Sites with Performance InformationNewsgroups with Web Performance ContentDisclaimerAcknowledgments for the Second Edition
Questions for the Browser SideIs Your Modem On?Is Your Modem Connected to Your Computer?Did the Other End Hang Up on You?Are You Sending Data over the Modem?Do You Have a Valid IP Address and Gateway Router?Is the Browser Hanging?Is Your Browser in “Offline” Mode?Can You Still Resolve Names?Is an Intermediate Router Down or Very Slow?Is the Remote Site Overloaded?Is the Remote Site Down?Is There a Mirror Site?Did You Already Get Most of the Page?Is Your MTU too Big, or too Small?Do You Need to Use a Proxy Server?Does Your Proxy Handle https URLs?Is There a Faster Proxy Server?Are You Being Blocked Deliberately?Are You Running too Many Programs?Do You Have Enough Memory?Is Your CPU Fast Enough?Do You Have Enough Bandwidth?Are Excessive Images Dragging Down Your Performance?Is Browser Startup Time Getting You Down?Are You Using a Slow Browser?Is Your Cache Big Enough?Are You Wasting Time Verifying Cached Pages?Is Java Startup Time Annoying You?Could You Benefit from Using a Specific ISP?Are You Surfing at Slow Times?Could Your Organization Benefit by Installing a Proxy Server?Questions for the Server SideIs Your Server Sleeping?Is Your DNS Server Overloaded?Do Your Images All Have Sizes and ALT Tags?Do All Your Applets Have ALT Tags?Are There Needless or Slow Redirects?Is Your Web Server Wasting Time on Reverse DNS Lookups?Is Your Web Server Retransmitting too Much?Are You too Far Away from Your Users?Is Your Server Network Connection Overloaded?Is Your Server CPU-Bound?Are You Short of Memory?Are Your Disks too Busy?Could You Benefit from Caching Services?Are You Suffering from Performance Bugs That Have Already Been Fixed?Is cron Killing Performance at Regular Intervals?Is Your Web Site Being Crowded by Other Processes?Are You Wasting Time with SSIs?Is Your Content Unncessarily Dynamic?Is Your Database Connection Pool too Small?Is Your Database Connection Pool Leaking?Are Your Hubs, Switches, and Routers Overloaded or Misconfigured?Do Java Processes Suddenly Stall?Are You Using CORBA, EJBs, or RMI?Are You Doing Excessive Logging?Are You Serving Content Directly from a Revision Control System?The Server Is Not at All Busy, but It’s Slow!Key Recommendations
Trade-offsState Versus ScalabilityReplication Versus SimplicitySynchronous Versus AsynchronousConnectionless Versus ConnectionfulPlanning Versus DoingProcedural Versus Object-OrientedElementsBrowserLoad BalancerDNS levelIP levelEthernet levelWeb ServerMiddlewareDatabaseExample Web Site ArchitecturesOne BoxStacksLayersLinux on the MainframeReal TimeTrendsSample ConfigurationsLow VolumeMedium VolumeHigh VolumeKey Recommendations
Do the Math . . .. . . But Trust Your Eyes More than the MathQuestions to AskHow Many HTTP Operations per Unit Time Do You Expect?What Is the Purpose of the Web Site?How Tolerant Are Your Users?Will You Provide Any Streaming Media?Will the Web Server Spawn Additional Processes?What Other Processes Need to Run on the Web Server or over the Network?What Sort of Scalability Do You Need?What Is Your Budget?How Available Does Your Site Have to Be?Can You Force Suppliers to Compete?Hey Kid, Want to Try a Browser?Other QuestionsHow Much Bandwidth Do You Need?Latencies Are More Important than BandwidthThinking About BandwidthEstimating Web Server Network BandwidthHow Fast a Server Do You Need?How Much Memory Do You Need?Memory for the Operating SystemMemory for httpdMemory for ContentMemory for CGIsKey Recommendations
Parameters of PerformanceLatency and ThroughputNetwork LatencyMeasuring Network Latency and ThroughputUsing FTPOther performance measuresUtilizationEfficiencyUsing a Shell ScriptUsing CUsing PerlMonitoring Web Performance Using PerlPlotting Results with GnuplotAn Example Monitoring Script in PerlThe PiecesAutomatically Generating Monitoring Scripts Using SprocketUsing a Relational Database to Store and Retrieve Your Monitoring DataStoring DataRetrieving DataMonitoring Machine Utilization with rstatStoring rstat Data in a Relational DatabaseUsing rstat DataGetting Data from the Database to Standard OutputGenerating Graphs Directly from an rstat DatabaseMonitoring Per-Process StatisticsUsing CGIs to Run ToolsUsing rstat to Trigger Deeper AnalysisUsing Telnet from PerlGenerating Graphs from ps DataMonitoring Other ThingsUsing Java for MonitoringMaking a System Dashboard Web PageDanger! Excessive Monitoring is Hazardous to Your Web Site’s HealthSNMPRMONARMOther ResourcesKey Recommendations
Load Test PreparationWatch Out for Clock ChangesWhy Is Production Performance Different from Test Performance?Trade-offs with Load Testing ToolsWriting Your Own Load Testing ToolsThe Timer ProblemLoad Testing by Excessive MonitoringSynchronizing a Load TestRandomizing a Load TestHelp, I Have to Stop My Load Test!Generating Network LoadBenchmark Specifications and Benchmark TestsWebStoneSPECweb99TPC-C and TPC-DProxy BenchmarksVendor-Standard BenchmarksCaffeineMarkOther ResourcesKey Recommendations
Using analysis.cgi to Find a BottleneckSnooping HTTP with SprocketLook at ConnectionsLog File AnalysisAverage Transfer SizeResponse Size DistributionHits per SecondVariable Load and Queue LengthWhen Exactly Are Hits Logged?Who Is the Most Common User?Which Process Is Mine?Who Is Using That File?What Files Are My Process Using?What Happens if the Database Is Hung?A Few More TipsKey Recommendations
Typical FailuresDisk FullProcess Out of File DescriptorsC Pointer ErrorMemory LeaksThread DeadlockDead Process Holding LockServer OverloadedLoad Balancer Fails to Detect Dead MachineSubnet FloodedOut of ptysDatabase Out of CursorsBad Device DriverHardware FailuresPower OutageAdministrator on Wrong ServerWildcards Including Wrong FilePermissions ProblemsPath ProblemsPatch ProblemsCascade of OverloadsMonitoring Causing FailuresRetries Cause Further FailuresCritical Tables Locked by AccidentUsing a Database Where Files Will DoProgram Fails to Reconnect to Database After FailureProgram Fails to Restart After RebootSplit-Brain SyndromeFirewall “Cleanup” Blocks Essential ServiceScreen Scraping Fails Because of Design ChangeDependenciesSmoothing OutagesKey Recommendations

HTTPS and SSLFirewallsBastion HostschrootKey Recomendation
Database Table Growing Without LimitReverse DNS Lookups Slows LoggingKinked CableDatabase Connection Pool Growth Limits PerformanceKey Recommendation
Principles of Performance TuningSometimes You LoseTo Measure Something Is to Change ItReading Is FundamentalThere Is No Free LunchReturns DiminishPortability Will Reduce PerformanceIncreasing Abstraction Will Reduce PerformanceSecurity Will Reduce PerformanceMemory Is HierarchicalCaches Depend on Locality of ReferenceI/O Is SlowInformation Is RelativeHardware Is Cheap, Software Is ExpensiveThe Goal of Tuning Is Simultaneous FailureBetter Is RelativeBits Are CostInternet Performance Degrades NonlinearlyTuning at the Highest Level Gives the Biggest GainsWhatever Occurs Once Is Likely to Happen Again SoonThe 80/20 RuleIt’s Not What You Know, It’s Whom You KnowPatterns of Performance ImprovementAmortizationCachingProfilingParallel ProcessingUsing What You KnowSimplicityKey Recommendations
How Browsers WorkTypes of BrowsersNetscapeInternet ExplorerOperaNeoplanetWebTVCelloMosaicLynxAmayaTangoThe Perfect BrowserBrowser SpeedBrowser Tuning TipsGeneral TipsUpgradeDo lessUse shortcutsIncrease cachesRebootMultitaskStop itUse Java plug-inBrowse in a new windowInternet Explorer TipsDon’t redraw while scrollingBrowse in a new processNetscape TipsPrestart JavaUse fewer colorsMake smaller buttonsNon-Browser Web ClientsGnutella and Peer to PeerKey Recommendations
Microsoft WindowsSystem ClutterSpecific Video DriversMemory and DiskSystem MonitorNetwork UtilitiesMTUMacintosh68K EmulationNetworkingMemory and DiskExtensionsUnixKey Recommendations
CPURAMCacheBusDiskIDESCSIFragmentationVideoMMXColors and ResolutionDrivers3D and Video ClipsVideo Card BenchmarksI/O PortThe UARTBIOSKey Recommendations
Forwarding and LatencyYour Modem, the Information DrivewayLatency and Throughput, Modem to ModemGetting SynchronousHardware CompressionError CorrectionLine QualityInternal Modems Are FasterUART Buffer OverrunsAT Commands and Dialing TimeBonding ChannelsISDNCable ModemsxDSLHigher Capacity Lines56K, T1, and T3Frame RelayATMSatelliteIntranetsPartitioningHardware for PartitioningEthernetPacket SnoopingNIC BuffersFast EthernetSwitched EthernetEthernet CablingNoiseNetwork Modeling ToolsThe InternetBypassing the InternetNAPsISPsPlacementISP performanceChoosing an ISPCo-location and mirroringRoutersPlacing BlameThe Future of the InternetInternet2NGIPTTsKey Recommendations
Power and ProtocolsFactors Affecting Network Protocol PerformanceFixed- Versus Variable-Sized Packets/Cells/FramesPiggybackingPipelining Versus ACK-ing Each PacketFull/half duplexErrorsNumber of HopsLayersThe Protocols of the WebARPPPPRouting ProtocolsIPTCPUse latest TCP patchesTCP parametersListen queuesRetransmission delayTIME_WAIT intervalAbort intervalMaximum segment lifetime and maximum connection rateKeepalive intervalReceive windowDelayed ACKSlow startMaximum Segment SizeMonitoring TCPT/TCPUDPDNSNFSHTTPStateless and connectionlessAsymmetricText-basedCompressionCache controlThe trailing slashHTTP 1.1 improvementsHTTP proxy request formatByterange downloadsHTTP and filesystemsFTPNNTPCORBAXKey Recommendations
Box on a WireGood I/OMultiple BussesFast DisksLots of MemoryScalabilityNetwork Interface CardBusMemoryRAM CharacteristicsCPUCPU ConstructionSymmetric Multiprocessing (SMP)Testing Sun SMP ScalabilityDiskDisk Architecture and ParametersIDEEIDESCSIFibre ChannelRAIDTypical Disk PerformanceFragmentationDisk Activity and PIDKey Recommendations
Unix and the Origin of the WebUnix FlavorsSolarisAIXDigital UnixLinuxIrixBSDMach OSSystem Calls Versus Library CallsProcesses and the KernelSchedulingKernel ContextUnix and httpdSolaris Network Cache and AcceleratorLinux khttpdReducing the Load on the OSProcess CreationAddress SpaceCopying in MemoryReclaiming MemoryThe FilesystemFilename LengthFull FilesystemsDirectoriesFilesystem CachingDirectory Name Lookup CacheFragmentationThe Windowing SystemVersions and PatchesConfigurable OS ParametersNumber of File DescriptorsNumber of ProcessesNetwork BuffersMemory LimitsFlushing FrequencyPriorityTimer InterruptUnix OS Monitoring ToolspsperfbarperfmeterperfmonrstatruphstattopxloadSystem Call TracersNetwork Snooping ToolsnetstatvmstatsarHow Many Connections Can My Server Handle?How Many Processes Can My Server Handle?How Quickly Can My Server Fork New Processes?Unix Versus NT as the Web Server OSNT Pros and ConsUnix Pros and ConsThe ExokernelKey Recommendations
The Evolution of Web ServersServers Spawned from inetdForking ServersThreaded ServersKeepalive ServersSystem Calls Made by a Web ServerHow Servers FailMemory LeaksConfiguring Apache and Netscape Web ServersConfiguring ApacheAllowOverrideBUFFERED_LOGSMaxClientsPersistent connectionsTurning off reverse DNSDo not restrict by domainSet FollowSymLinksFancyIndexingUse specific index filesMaxRequestsPerChildSome notes on sizing ApacheUsing mod_statusFor more informationConfiguring NetscapeNumber of processesNumber of server threadsTurning off reverse DNSPersistent connectionsFile cachepwfileCGI timeoutRqThrottleDisable extraneous featuresUsing perfdumpFor more informationOther ServersBoaIISJava Web ServerJigsawNCSAZeusMissing FeaturesProxy ServersHierarchical CachesKey Recommendations
Size MattersAs Good As It GetsCaching and DifferencesHTML and CompressiongzipPerformance Tips for HTML AuthorsMake It Easy on the ServerMake It Easy on the NetworkMake It Easy on the BrowserMake It Easy on the UserWatch Out for Composition Tools with a BiasKeep Up with the State of the ArtUse an HTML ValidatorThe Document Object ModelGraphicsWeight WatchingConsolidationReusePsychologyFormatsAudioVideoKey Recommendations
ProgrammersCGI ProgramsCGI Internals and Performance ProblemsGeneral CGI TipsInfinite LoopsHow to find and kill infinitely looping CGIsRunawaysGuarding against infinitely looping CGIsDon’t Keep the Customer WaitingPush State or Processing into the BrowserCookiesJavaPreprocess Queries and Cache the ResultsSmall Is BeautifulScaling IssuesBreak Up Long Forms into Several Small FormsServer DNS LookupDebug and OptimizeCGI Language-Specific Optimization TipsShell ScriptsPerlCDaemonize ItCGI Database Access PerformanceLoggingNSAPI and ISAPIDOMJSP, ASP, PHPKey Recommendations
Java Will Never Be Good Enough for GUI ApplicationsJava Is Good Enough for the Server SidePerformance Problems Intrinsic to JavaArray Bounds CheckingBlocking Network I/OBytecode InterpretationBytecode VerificationDynamic Method BindingGarbage CollectionIndirectionsi18n & l10nOOStack-basedSynchronizationThreadingCoding TipsUse Good AlgorithmsKeep Short Inheritance ChainsUse Stack VariablesMerge ClassesUse Java LibrariesDo Not PollFinalize MethodsCreate Fewer ObjectsBeware of Object LeaksConsider Not Using Accessor MethodsUse Compound OperatorsUse Int IncrementsBe Conscious of Access Speed for Different VariablesLocal Variables are Faster than Class VariablesClass Variables are Faster than Array AccessesUse Faster Types as Loop IndexesUse Native MethodsUse Network TimeoutsBuffer Network and File I/OUse Sockets Rather than URLsUse UDPUse ThreadsUse notifyUse Synchronization SparinglyKeep Synchronized Methods Out of LoopsWatch Out for the Mother ThreadCounting Down May Be Faster than Counting UpReduce the Number of StringsUse String Buffers or ArraysWatch Out for Slow FontsKeep the Paint Method SmallDouble Buffer Where Possible for Smoother GraphicsUse the Runtime to Do Error Checking for YouAvoid Time ConversionsBe Wary of RMI, EJBs, and CORBACompilers-OProfile Your CodeJVMPIDecompilersOS-Level Profiling ToolsJITsStatic CompilersVirtual MachinesRuntime Options-verbosegc-noverify-Xmsn and -XmxnMake Eden Big Enough-trainUse Native ThreadsUse .jar FilesJava Plug-In-start_ javaJava ChipsJava BenchmarksProblems with BenchmarksWeb Sites with Java Performance InfoKey Recommendations
Do You Really Need a Relational Database?When You Need a Professional DatabaseAlternativesPerformance TipsPrepared Statements and Bind VariablesDenormalize TablesDo Not Create Cursors in LoopsDirect ConnectionsMain Memory DatabasesMultiple TiersConnection Pool ConfigurationQueriesIndexesRow-Level LockingCombined Web Server/DatabasesHow Many Connections Can Your Database Handle?When the Database Is OverloadedAnalysisKey Recommendations
Problems with Commercial ToolsMonitoring ToolsLoad Generation ToolsPreloadersNetwork OptimizersMTU TunersOptimal Application ExpertIP Traffic Management ProductsContent CompressorsCondenserT/X 2100 Series from Redline NetworksHybrid Development Tools/DatabasesJava Profilers and OptimizersCaching ServicesProfessional ServicesLoad BalancersModeling Tools

Content preview from Web Performance Tuning, 2nd Edition

Chapter 17. Server Operating System

The operating system sits between the hardware and the web server software, translating the web server’s request for services into hardware actions and delivering data from the hardware back up to the web server. The server hardware has a maximum performance that is fixed by its physical specifications. You can approach this maximum performance by appropriate configuration of the operating system. The question for web servers is how to configure the operating system to take requests from the network, find the correct file to return or run the correct program, and push the result back out to the network, all as fast as possible.

Except for a brief comparison of Unix versus Windows NT towards the end, this chapter focuses squarely on Unix. A survey at http://leb.net/hzo/ioscount/ from April of 1999, admittedly a bit old but the most recent I could find, found that about 75 percent of all web sites run on Unix of one kind or another, with Microsoft OS’s accounting for another 24 percent. Among Unixes, the most common were Linux at 27 percent, Solaris/SunOS at 20 percent, and BSD at 16 percent.

Unix and the Origin of the Web

Networking has been central to the development of Unix. Unix was originally developed around 1970 as a research project by AT&T Bell Laboratories, building on multiuser and multitasking ideas from the Multics government research project of the 1960s. Since AT&T was barred from selling software because it was the U.S. telephone ...