SharePoint world of ECM and Information Management : Performance

Use the right data for SharePoint testing

Michael — Wed, 09 Sep 2009 17:17:18 GMT

Thanks to Dario Mratovich, Microsoft Consultant, for his awesome recommendations of preparing the testing data. Citing the section from his “Capacity Planning Testing for SharePoint 2007”

Make sure you have adequate sample data. This tends to be a very common stumbling block – sites are built with only a tiny percentage of content that production will have. Not enough sites, not enough content, not exercising a broad enough sampling of your dataset, not enough users – these can all fatally influence your test results.

Another common problem is having enough content for a reasonable search corpus. What many people try doing is uploading the same document many times – sometimes hundreds or thousands of times – and think that if it has a different file name then it will be okay. Unfortunately in that scenario the search duplicate process can start taking significantly longer than it otherwise normally would, so this too can unfairly reduce your query throughput.

A document that is uploaded multiple times will affect the way that SharePoint performs duplicate detection: where SharePoint’s search calculates a hash based on the contents of a document – it doesn’t look at the filename! So uploading a document 30,000, even with a different filename, will cause the search retrieval to become slower and slower as SharePoint tries to resolve duplicate documents.

You will need tools in all likelihood to populate sample data. Some tools you can start with are on CodePlex at http://www.codeplex.com/sptdatapop. You will probably end up writing additional tools for other data population tasks, or possibly to work in combination with these tools.

Using PowerShell for scripting the creating of objects and data for testing is also very useful.

SharePoint Tip #39. Do you know “how to measure performance of virtualized environment”?

Michael — Fri, 12 Jun 2009 14:07:12 GMT

Virtualizied environment is very common for SharePoint farm. One of the most important factor of such farms is optimization that differs slightly from optimization of the physical environment . But firstly, we need to know how good our farm operates, measuring different parameters, before trying optimized it.

Consider to use use the following performance counters to measure the most important parameters that affect performance:

Processors utilization
[Host]: “\Processor(*)\% Processor Time”
[Host] [Guest]: “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time”

Results: <60% healthy, 60%-89% warning, >90% – critical performance;
Memory performance
[Host] [Guest]: “\Memory\Available MBytes” will show the amount of physical memory available to processes running on the computer, as a percentage of physical memory installed on the computer

Results: (free memory available): >50% healthy, 10% warning, <5% critical

[Host] [Guest]: “\Memory\Pages/sec” - the rate at which pages are read from or written to disk to resolve hard page faults. To resolve hard page faults, the operating system must swap the contents of memory to disk, which negatively impacts performance. A high number of pages per second in correlation with low available physical memory may indicate a lack of physical memory

Results: <500 healthy, 500-1000 warning, >1000 critical
Disk Performance
[Host] “\Logical Disk(*)\Avg. Disk sec/Read” or “\Logical Disk(*)\Avg. Disk sec/Write” to measure disk latency

Results: < 15ms healthy, 15ms-25ms warning, >25ms critical.
Don’t forget to measure performance of logical disk versus physical
Network Performance
[Host] “\Network Interface(*)\Bytes Total/sec” will provide the percentage of network utilization is calculated by multiplying Bytes Total/sec by 8 to convert it to bits, multiply the result by 100, then divide by the network adapter’s current bandwidth.

Results: <40% healthy. 40%-65% warning, >65% critical

[Guest] “\Network Interface(*)\Output Queue Length” measures the number of threads waiting on the network adapter.

Results: 0 is health, 1-2 is warning and >2 is critical
[Host] [Guest] Free disk spaces – have enough available space to avoid SharePoint move and delete temporary files, what could affect the performance. SharePoint uses additional space for for caching and other internal processes.

Result: >25% is healthy

Sources: 0, 1, 2, 3, 4, 5

Have anything to add?! Send your tips to be published via this form.

SharePoint Tip #29. Do you know “why to split large collaboration site between different site collections”?

Michael — Sun, 12 Apr 2009 07:00:00 GMT

SharePoint functionality is build on the top of SQL Server and has a tight cohesion with SQL Tables. Any user’s actions on the SharePoint site lead to the data entries manipulations across several tables.
Unfortunately, SharePoint implemented in the way to have few large tables rather then several small tables, and it stores a lot of contents in one single table. Such approach has a side-effect in table locks, which might hinder the performance of SharePoint sites if you are not aware about such table-centric design

With this in mind, consider splitting the content of large Collaboration site between different Site Collections. The reason for this that Site Collection is implemented as a huge single table.

Let’s imagine that you have site collection with 200 sub-sites, and some of user’s action force the transaction in SQL Server. Such action will lock Site Collection table, thus all 200 sites will be locked as well. In this case all users can’t interact with the SharePoint and will be waiting when that action competes.

There is no workaround for this, and the only way to track such issue is checking how your SharePoint sites performs – it case number of locks your will see noticeable performance degradation. Monitor SQL locks against SharePoint tables, to find such cases.

Update: consider splitting your site collection to be less then 100Gb-150Gb, otherwise you will have lots of locks in site collection table. The best tools to split site collection and move content across contentDB is "SharePoint Administration Toolkit"

Have anything to add?! Send your tips to be published via this form.

SharePoint Tip #25. Do you know “How to calculate Content DB grow”?

Michael — Wed, 18 Mar 2009 13:57:35 GMT

SharePoint content databases grow very rapidly when used in document imaging system or in file share replacement scenarios. With this in mind it is important to consider growth and overhead factors when determining how much content will eventually be stored in a given content database.

Use the following formula to calculate Content DB size.

Low: 1.2 * [Raw Storage Size] = ContentDB Size
High: 1.5 * [Raw Storage Size] = ContentDB Size

If possible, separate each data file to exist on separate logical units consisting of unique physical disk spindles

Source

Have anything to add?! Send your tips to be published via this form

SharePoint Tip #24. Do you know “How do I tune the MOSS Object Cache for performance”?

Michael — Fri, 13 Mar 2009 03:44:11 GMT

Steve Sheppard published nice recommendation regarding optimizing the MOSS Cache performance and how to measure the performance.

Citing the tips section from his post

We feel this level of cache performance will meet an economical customers performance needs without unduly sacrificing their limited memory resources. The steps to achieve this are fairly straightforward and must be applied in an iterative fashion until the desired level of performance is achieved. The recommended steps are:

Start with the default cache settings of 100MB on the site collection.

Capture at least 8 hours worth of performance data from the WFEs while the system is under a typical load using the "SharePoint Publishing Cache/Total number of cache compactions" counter. Since we are interested in tracking compactions per hour it is acceptable to capture this data at 1 minute intervals.

After analysis of the data, if we are exceeding the threshold for acceptable cache compactions we will need to add an additional 50MB to the Maximum Cache Size value and run the test again.

We should continue this process until such time as we have achieved an acceptable cache compaction rate.

You can find the full post there

Have anything to add?! Send your tips to be published via this form

SharePoint Tip #22. Do you know “how to manage large list performance”?

Michael — Sun, 08 Mar 2009 12:18:40 GMT

When a list or library has a large number of items, you must carefully plan its organization and how users need to access the data to help preventing adverse effects on performance.

For best performance, do not exceed 2,000 items in a list level. For example, the root of the list or a single folder. In case if you must create and browse large lists, use the following best practices.

1) Create Indexed Colum

An index on a column enables Microsoft Windows SharePoint Services 3.0 to quickly analyze the data in that column, even when working with thousands or millions of items

2) Use only one Indexed Column in view

3) First column that is used to filter the view must have an index

4) Change the default view of the list to a customized filtered view that follows these recommendations:

•    The view returns fewer than 5,000 items.
•    The first column that you use to filter the view has an index that sufficiently reduces the total number of items returned.
•    The view displays only those columns that are absolutely required.
•    The view includes as few lookup columns as possible. Each lookup column in a list that is included in a view causes an additional join and additional calls to the database.

Source

Have anything to add?! Send your tips to be published via this form

SharePoint Tip #18. Do you know “why to use dedicated Web Server for crawling”?

Michael — Thu, 05 Mar 2009 02:21:00 GMT

By default, Office SharePoint Server 2007 uses all of the front-end Web servers in the server farm to crawl content in the same server farm. The index server sends requests to each front-end Web server in the farm. The front-end Web servers get the requested content from the SharePoint sites in the farm and forward that content to the index server for indexing.

This works well for small-to-medium-size organizations. Large organizations, however, tend to crawl more content than small organizations. This can translate into hundreds of gigabytes or even terabytes of content that is being crawled and indexed, what places a heavy load on the front-end Web servers. Such load cause negative impact on the performance (CPU usage and up to 50% of trafic) of all front-end Web servers in your server farm.

To improve the overall performance in large farm configure a dedicated Web server for crawling content, especially if you are crawling a server farm that contains more than 500 gigabytes (GB) of content or if you are crawling content over the WAN
Consider removing this dedicated server from NLB routing

Source

Have anything to add?! Send your tips to be published via this form