Using Windows Azure for smarter BLOBs storage

Posted on April 6, 2012 by Iqbal Khan

Windows Azure Platform consists of three product brands. These are Windows Azure (an operating system providing scalable compute and storage facilities), SQL Azure (a cloud-based, scale-out version of SQL Server) and Windows Azure AppFabric. Unstructured data or BLOB storage is provided as part of Windows Azure Storage services. Some of the immediate benefits associated with SharePoint BLOB storage content in off-premises Windows Azure are:

1. Fault-tolerance: Replication of BLOBs to multiple locations makes Windows Azure BLOB storage highly resilient.
2. Cost Reduction: Operational costs associated with acquisition, operation and maintenance of storage hardware are immediately eliminated.
3. Flexibility: BLOBs can be consumed from anywhere.
4. Scalability: Windows Azure can offer virtually limitless storage space at minimal cost.
5. Seamless Integration: With solutions such as StorageEdge, the integration of SharePoint with Windows Azure based storage just becomes very simple (shown later in the article).

However, most IT Pros show immediate concerns for the control, security of the content, data access performance, in-sync backup/restore and storage service uptime. Since Azure-based storage means that you offload the BLOBs from SharePoint, lack of control regarding the management of storage is a natural consideration.

Optimize SharePoint Performance and Storage - Free Download

The answer is to adopt a hybrid deployment approach, that is, to keep SharePoint on premise along with most commonly used BLOBs, and then store the older and less frequently used BLOBs in Windows Azure which can provide you significant cost reduction. Use of StorageEdge can help you gain additional comfort through a number of essential features while you move older documents out of your on premise infrastructure (discussed in the subsequent section).

SharePoint BLOB storage can be externalized to Windows Azure Storage using RBS based BLOB externalization facilities of StorageEdge.

Figure 1: SharePoint BLOBs Storage on Windows Azure

StorageEdge allows the enterprises to move SharePoint BLOBs from SQL Server to virtually any storage option including Windows Azure. StorageEdge helps overcome many of the risks and considerations associated with cloud storage. It allows you to create a healthy mix of data storage with some of the data hosted locally while some data offloaded to the cloud.

StorageEdge lets you externalize SharePoint BLOBs that you desire to Windows Azure Storage. In fact, with StorageEdge you can use a number of filters to automatically identify the right candidates for externalization. StorageEdge lets you create storage profile for Windows Azure to externalize the SharePoint BLOBs there. Following figure shows the GUI to configure a storage profile for externalizing content to windows Azure Storage.

Figure 2: Configure a Storage Profile for Windows Azure in StorageEdge

When you configure the storage profile for windows Azure, StorageEdge requires you mainly to set a Windows Azure Storage connection string containing the account name, key given by Microsoft and a container property.

Then, you can specify different types of criteria for externalizing BLOBs such as Age and Versions. Once the storage profile has been configured, you can immediately see the BLOBs being externalized to Windows Azure Storage.

Some of the additional important features you can use with Windows Azure Storage are:

1. Encryption of Content: The BLOBs are encrypted before writing to external storage. StorageEdge supports encryption on a storage profile thus providing first layer of security to the externalized BLOBs. It supports DES-64bit or AES-128bit encryption.

Figure 3: Encryption Settings in StorageEdge

2. File Shredding: The tool responsible for externalized BLOBs management must make sure all permanently deleted content is irrecoverable using any third-party tools. StorageEdge has a fool-proof file shredding mechanism coming into play when the BLOB is permanently deleted from SharePoint.

Figure 4: Shredding in StorageEdge

3. Compression: StorageEdge provides for compression of BLOBs using GZip.
4. Encrypted File Names: The BLOB management tool must be such as to ensure that the file names are encrypted. This becomes especially important when you move the content to a storage media which is out of your enterprise such as in the cloud.

So, StorageEdge takes care of your hybrid SharePoint infrastructure management needs with provision of enough control and flexibility. So having Windows Azure Storage as the mainstay of your BLOB archiving strategy, not only reduces storage costs, but it also improves SharePoint performance because the most active storage no longer contains a huge amount of documents that can potentially inundate it.

Download StorageEdge | StorageEdge Details

Posted in SharePoint Storage | Leave a comment

Organize SharePoint Content on Remote Storage

Posted on April 6, 2012 by Iqbal Khan

With all and sundry blowing the trumpet of externalizing the BLOBs, I decided to explore one of the most important consideration for the SharePoint administrators – the management aspects of the externalized BLOBs. And as expected, StorageEdge didn’t disappoint me at all, rather it clearly stood out in its coverage of even the minute features which others found trivial. Let us go straight into some of its important features regarding management of BLOBs on remote storage.

Files and Folder Structure on Remote Storage

With StorageEdge, you can have the same folder structure on remote storage that you have in SharePoint. You can use auto generate folder and set a limit on the number of files per folder. Similarly, you can use same file names on remote storage as you have in SharePoint. This makes searching of the content easy and an experience which exactly matches to that of a local storage.

Figure 1: Files and Folders Management in StorageEdge on Remote Storage

You can identify whether the BLOBs are to be externalized in folders or not. It will help you to define the file name and the folder structure for your content. You can keep the data in the same file structure as in SharePoint. The options available are:

Optimize SharePoint Performance and Storage - Free Download

  • “Do not create folders” externalizes the data in a flat structure; no folder hierarchy is defined.
  • “Use SharePoint folder names” externalizes the data in the same hierarchy as on SharePoint.
  • “Auto generate folders” auto-generates the folder/(s) with date time stamp.

“Use SharePoint file names” keeps the file name same as it was on SharePoint.

Encryption of Content

The BLOBs are encrypted and written to any external media such as cloud storage subsequently. StorageEdge supports encryption on a storage profile thus providing first layer of security to the externalized BLOBs. It supports DES-64bit or AES-128bit encryption. 

Figure 2: Encryption Settings in StorageEdge

File Shredding: StorageEdge has a fool-proof file shredding mechanism coming into play when the BLOB is permanently deleted from SharePoint. It ensures that all permanently deleted content is irrecoverable using any third-party tools. 

Figure 3: Shredding in StorageEdge

Compression: StorageEdge provides for compression of BLOBs using GZip.

 

Figure 4: Compression in StorageEdge

Encrypted File Names: File names encryption becomes especially important when you move the content to a storage media which is out of your enterprise such as in the cloud. StorageEdge hides your actual filenames, if you choose to do so, to keep your content safe.

StorageEdge promises a dependable management of your externalized content on remote storage. It not only ensures substantial cost savings with respect to you storage infrastructure, but also relieves you of the concerns of managing large volumes of data, some of which may even lie out of your premises.

Download StorageEdge | StorageEdge Details

Posted in SharePoint Blob Caching | Leave a comment

Move from EBS to RBS

Posted on April 6, 2012 by Sameer Shah

SharePoint has seen such wide-scale adoption and enough empirical evidence is now available to establish that BLOB externalization outperforms in-SQL storage. BLOB storage in SQL Server hampers the overall performance of a SharePoint infrastructure. Microsoft first introduced External BLOB Storage (EBS) and then Remote BLOB Storage (RBS) in SQL Server 2008 R2, only as interface specifications, to let you take BLOBs out of SQL Server. StorageEdge fills the gap nicely as it provides enterprise-grade implementations for both EBS and RBS to offload BLOBs to inexpensive external storage.

A detailed comparative analysis between EBS and RBS has been provided in my earlier blog here. Primary difference is that RBS is implemented in SQL Server 2008 and has no direct relation to SharePoint, whereas EBS has been provided as a layer in SharePoint stack from where it talks to SQL Server. Moreover, RBS is not unique to SharePoint as it is available to any application using SQL Server, whereas EBS is SharePoint specific. RBS has Managed interface whereas EBS’s is unmanaged. And RBS has more permanence, as EBS is scheduled to be phased out in the next version of SharePoint; EBS has been deprecated in SharePoint 2010 already. So, for these reasons, RBS is being widely adopted by SharePoint administrators and are shifting their content to RBS.

EBS, for having been around for a good number of years, has remained in use for externalizing large volumes of BLOBs to inexpensive tiers of storage. With RBS, there is a viable alternative available to the SharePoint administrators to which they might choose to migrate. However, one consideration that can stop them to convert from EBS to RBS is the actual movement of such large caches of BLOBs from EBS to RBS. Certain providers require you to bring the content into SQL Server first, configure an RBS provider on SQL Server and then externalize again. This presents an impractical approach, as it is a huge challenge for TBs of data to achieve and shall have negative impact on the overall performance of the SharePoint infrastructure. So, traditional providers do not provide you with an easy conversion path from EBS to RBS.

Optimize SharePoint Performance and Storage - Free Download

But StorageEdge does. Its tightly integrated architecture lets you to move from EBS to RBS, without actually moving your BLOB content. If your BLOBs have been externalized through EBS, and you desire to use move to RBS, you can use StorageEdge to convert to RBS with BLOBs being retained on the same external location. With StorageEdge, you are only required to enable RBS on the corresponding content database and specify the same storage profile settings and storage location as you have for EBS.

Figure 1: Convert EBS to RBS in StorageEdge

StorageEdge lets you do this in two easy steps:

  1. Select the Storage Profile for the SharePoint web application you want to convert to RBS.
  2. Select “Convert to RBS” option from drop down menu as shown below.

StorageEdge even lets you schedule this conversion job at a time, when your system resources are available, to avoid hampering SharePoint’s performance at peak usage times.

Figure 2: Schedule a Conversion from EBS to RBS in StorageEdge

So in a nutshell, StorageEdge makes life a lot easier whenever you choose to convert from EBS to RBS.

Download StorageEdge | StorageEdge Details

Posted in BLOB Externalization | Leave a comment

To Externalize SharePoint BLOBs or Not?

Posted on March 8, 2012 by Sameer Shah

A SharePoint administrator or an application designer is often faced with whether to externalize the BLOBs or not, and what architectural parameters to consider while making this decision. With the availability of frameworks such as EBS and RBS from Microsoft and enterprise solutions built around them such as StorageEdge, a lot of buzz abounds the SharePoint circles on offloading the BLOBs out of SQL Server. But there are arguments on both sides regarding whether to externalize the BLOBs or not. People who advocate taking an all out externalization path base their argument on the following benefits obtained through externalization:

1. Reduce database size and management – moving BLOBs out of the database reduces its size by as much as 95%. And, this alleviates various database administration problems that come with storing large number of BLOBs in it.

2. Reduced cost storage- moving the BLOBs out of expensive teir-1 storage to less expensive, non-transactional storage introduces visible cost savings. Read here for a primer.

3. Better storage management – hierarchical storage on multiple tiers yields proper BLOBs management and archiving based on multiple criteria. Read here more. And you always have the option of exploiting the benefits of underlying storage platform as well.

4. Not all data is alike – You cannot treat all your data alike. It is an unwise proposition to keep stale or infrequently-accessed data in transactional tiers of storage.

5. Performance improvement in overall SharePoint access – SQL Server, despite being a high-performing resource manager, performs inadequately when it comes to large streams of unstructured data. SQL doesn’t support a data file of more than 200 GB and needs to be split. Write operations are particularly latency-inducing as the BLOBs are written first to transaction log and then in the table. The net effect is a poorly-responsive infrastructure.

Optimize SharePoint Performance and Storage - Free Download

Contrary to the above, people who believe otherwise base their arguments on:

1. Management of multiple data sources – Since due to externalization, the content is separated from the metadata, it induces challenges of management and synchronization.

2. Storing files in SQL Server is easier from a back-office management perspective.

3. Backup and restore challenges – Due to introduction of multiple storage platforms, challenges of backup and restore and managing consistency across all crops up. Read here more.

Empirical evidence has established that the advantages of externalization out-weigh the advantages of in-SQL BLOB storage, for most of the SharePoint installations, which essentially is a document-centric facility with multiple types of content involving varying sizes and characteristics. I have seen customers enjoying an immediate cost saving with respect to their storage investments ranging in 7 to 8 figures.

However, not all content is a good candidate for externalization. There are a number of architectural parameters, which you need to carefully consider, before deciding “which” of your BLOBs may be externalized. Read here for a discussion on which of your unstructured data is a good candidate for externalization and under what circumstances.

Download StorageEdge | StorageEdge Details

Posted in Benefits of BLOB Externalization, Externalize BLOBs | Tagged , | 1 Comment

Which BLOBs to Externalize?

Posted on March 8, 2012 by Sameer Shah

In an earlier debate here, I had talked about the fact the although there is a very strong case in favor of externalizing the BLOBs out of SQL Server to inexpensive tiers of storage for most of the SharePoint installations, there are certain BLOBs and architectural situations still which would require you to act otherwise. “Which BLOBs to externalize?” generally gets a traditional response, that prevails among the practitioners. That is, the databases are better suited for small objects while filesystems are suitable for large objects. SQL Server is particularly optimized for 8KB or less structured data, but SharePoint usually carries documents larger in size. For small, structured data, database queries are faster than opening files and opening large files on filesystem is faster than accessing large BLOBs in database.

This approach may work for many typical SharePoint installations, but not many quantitative benchmarks are available to establish it across all types of SharePoint architecture, SharePoint storage and SharePoint content. Moreover, boundary between “small” or “large” is pretty fuzzy as well. Not all SharePoint content being a good candidate for externalization, you should consider a number of parameters, before deciding “which” of your BLOBs may be externalized. So, naturally a very important parameter is the BLOB size.

Optimize SharePoint Performance and Storage - Free Download

A research report, a little old but still relevant, from Microsoft Research available here summarizes the correlation between BLOB size and its appropriate storage location in the form of following benchmarks:

• BLOBs less than 256KB are more efficiently stored in the database
• Files larger than 1MB are more efficiently stored in the file system
• Content between 256KB and 1MB could be stored in either, but it is better to leave it on the filesystem.

So, generalizing, firstly if you have very few large (> 256KB) documents, you might want to retain them on SQL Server provided that the total size doesn’t exceed 200GB. However, in a typical SharePoint environment, this is hardly the case. Secondly, If more than half of your BLOBs are large (>256KB), then externalize all of them. The third situation is more likely to occur where you have a variety of sizes in SharePoint repositories. In an ideal case, you must have a way to filer content based on both individual file sizes and the need to keep the overall data file size below 200GB.

In a medium to large scale SharePoint environment, there can be millions of documents with varying sizes and accumulating in TBs. In such a case, you should consistently and reliably be able to separate the documents which you want to externalize from the ones which you want to retain inside the SQL Server in an configure-once, use-repeatedly setting. In fact, this is only possible if you use a solution tightly integrated with SharePoint and is more than a standalone EBS or RBS provider such as StorageEdge. StorageEdge allows you to filter content based on a number of criteria when you want to externalize it. It has size-based, filename-based and author-based filtering options to externalize the contents based on the benchmarks provided earlier in this discussion.

BLOB Filter

BLOB Filter

Figure 1: Filters in StorageEdge for Content Externalization

In addition to the above-mentioned size based benchmarks, which should work well for most of the SharePoint systems, you must also consider that within these ranges, how much of the BLOB payload is write-intensive, and the BLOB retention age.

Download StorageEdge | StorageEdge Details

Posted in BLOB Externalization, SharePoint Content | Tagged , , | Leave a comment

How to Archive SharePoint Documents for Compliance

Posted on January 31, 2012 by Sameer Shah

Organizations are often faced with the challenge of storing and retaining documents for compliance purposes for varying durations. Dealing with SEC, HIPAA, SOX, or DOD compliance requirements call for a reliable archiving and retention storage architecture. Administrators cannot afford any tampering with those documents.

SharePoint is primarily a document-centric solution and offers excellent collaboration and publishing features. However, it can present significant challenges in meeting the regulatory requirements for storage, retention and discovery. SharePoint’s built-in storage architecture fails to meet the level of compliance needed by the organizations. However, with the availability of RBS providers in your SharePoint infrastructure, these shortcomings can be overcome by separating BLOBs from the content metadata and exploiting vast possibilities of external storage and retention.

Optimize SharePoint Performance and Storage - Free Download

There are different compliance requirements scenarios. One of the key among them is that you have to archive and retain certain types of documents and BLOBs in separate isolated storage. Archiving BLOBs in separate storage for compliance purposes ensures that these BLOBs are not accidentally deleted from SharePoint by users because a copy of the documents is kept in this separate retention archive from where they can be restored by SharePoint administrators.

Similarly, in many situations, you have documents that you must preserve for a predetermined period for regulatory compliance reasons and then they have to be automatically deleted from SharePoint. An example of this is current contracts and agreements in a financial, insurance, or other similar industry. On the other hand, certain compliance frameworks might require that some documents be automatically deleted after certain period of time has elapsed such as financial information that you don’t want to keep for legal purposes.

SharePoint Compliance

SharePoint Compliance

Figure 2: BLOB Retention for Compliance Purposes

However, not all RBS providers for SharePoint allow you the “archive” and “retain” capability at the same time; for such providers, “Archive” only means moving the BLOBs from storage tier to another.  Any SharePoint RBS provider that gives you this retention plus archiving capability must let you specify a separate storage, which is an archive to be used ONLY for retaining documents. StorageEdge has been built keeping in view these important considerations. It allows you to store compliance-related content enabling comprehensive management of SharePoint content throughout its lifecycle. It lets you create a retention archive which is protected for all regular SharePoint operations, either in the form of WORM (Write-Once-Read-Many) or only for the SharePoint administrators to access it.

Moreover, a good RBS provider must give you the ability to specify filters on which documents to put in retention archive. These filters could be on name, extension, size, content type, ownership, and more. The main idea is to give you real control over specifying which documents you want to archive and which to retain. StorageEdge gives SharePoint administrators flexibility and control over when, where, why, and how SharePoint content BLOBs are stored through application of filters. You can store BLOBs on multiple tiers of storage, according to a wide range of criteria including size, age, number of retained versions, changes in metadata or state (e.g. completed), file type (e.g. PDF, TIF, etc), content type and so on, meeting both operational and compliance needs.

Download StorageEdge | StorageEdge Details

Posted in SharePoint Storage | Leave a comment

How to Reduce SharePoint Storage Cost with BLOB Offloading

Posted on January 31, 2012 by Sameer Shah

Data loses its value as it gets older. The reference data which is less frequently used grows exponentially as compared to active data with the passage of time. Treating all this data alike has a negative impact on the costs of storing it as it leads to higher costs and less than optimal utilization of your storage infrastructure.

For a SharePoint environment, this problem is further aggravated by BLOB storage. Coming from a SharePoint Administrator background, I have seen numerous installations of SharePoint that suffer from issues related to management of large volumes of BLOBs, whether new or old. So, despite huge investments in the storage infrastructure, the users end up with a poorly-performing-yet-high-cost SharePoint storage infrastructure.

So, you need to align your storage options across two lines of action. Firstly, adopt the most appropriate storage tier for the data being stored and secondly, keep the data growth in line with its frequency of access.

BLOBs generally consume 95% of the SQL Server storage in SharePoint.  Although SQL Server is a high performance resource manager, when it comes to large binary data streams, it no longer remains a viable option. In SharePoint, a large proportion of content is documents stored as BLOBs. And, unfortunately SQL Server, like all other relational databases, was not designed to handle such large amounts of unstructured data. As a result, a number of problems usually occur.

Optimize SharePoint Performance and Storage - Free Download

As the data file size grows larger than 200 GBs, SQL Server performance degrades considerably which results in a poorly-responsive SharePoint environment. Similarly, storing all the BLOBs in SQL Server is not cost effective as SQL Server storage is typically very expensive. So, it is essential that you should store BLOBs on a lesser expensive storage tier that can manage unstructured data.

Similarly, considering the high costs that the storing of older BLOBs in high-end storage can cause, it is unwise to treat all the BLOBS alike in terms of its storage.

The above situation calls for simply offloading the SharePoint BLOBs payload from expensive, transactional storage to less expensive external storage as it will give you visible cost savings. An effective approach for the storage of externalized BLOBs can be to structure the external storage as a hierarchy of multiple tiers in the form of a hierarchical storage management system (HSM), with one storage tier corresponding to one age-based category of BLOBs.

StorageEdge precisely lets you do that. It has been built keeping the above-mentioned BLOB storage consideration in perspective. It provides multi-tiered storage that allows you to offload your active content in the most expensive storage and archives older content out to less expensive storage.

multi-tiered storage

multi-tiered storage

Figure 1: multi-tiered storage

So, by having an intelligent, reliable facility to offload SharePoint BLOBs in multi-tiered storage, you are able to control your storage cost in alignment with the business needs as you are able to grow you storage options in an incremental manner. You can expect great cost savings if your storage system can continue to move older BLOBs to less expensive storage tiers.

Surprisingly, it also improves SharePoint performance because the most active storage (meaning your Tier-1) no longer contains all those huge amount of documents that would have overwhelmed it. In a nutshell, externalizing BLOBs from SharePoint is not an option; rather it is a necessity for the best use of your SharePoint to perform. StorageEdge lets you achieve that with an absolute ease.

Download StorageEdge | StorageEdge Details

Posted in SharePoint Storage | Leave a comment

Impact of Job Throttling On SharePoint Performance

Posted on October 5, 2011 by Sameer Shah

Externalizing BLOBs is the nicety that can enhance the performance of SharePoint remarkably. However, dealing with TBs of content requires that tasks being performed on this large volume of data don’t hamper the performance of the SharePoint infrastructure. Consider a typical SharePoint environment where the total size of content can easily reach in TBs.  An archival job from one tier of storage to another or such other BLOB transfer activity can easily choke the SharePoint servers. At peak times, if these jobs continue to run, the capacity of the SharePoint infrastructure to service the user requests is seriously compromised.

SharePoint Job throttling is the technique of regulating the performance of these tasks in any system. It can be used in situations where it is required to limit the amount of data that can be transferred in a unit of time. Long running jobs such as Migrate, Revert, Garbage Clean, BLOB Transfer, and EBS to RBS Convert etc are heavily I/O bound jobs having a potential to considerably slow down the performance of WFEs. So SharePoint job throttling is the mechanism which allows the execution of a job / task to be configured to consume system resources at various levels at different points in time. This arrangement ensures that SharePoint WFEs dedicate their maximum resources to servicing the user requests during peak hours and let these resources be consumed in completion of heavy, I/O bound jobs during the non-peak times.

Optimize SharePoint Performance and Storage - Free Download

StorageEdge provides comprehensive SharePoint job throttling features. When these jobs / tasks are run, various time slots can be configured to make sure that SharePoint application is not under continuous job processing stress.

Job Throttling Settings

Job Throttling Settings

Figure 1: Job Throttling Settings in StorageEdge

StorageEdge provides you the facility to configure these tasks / jobs to be run at different speeds during different timings. Different tasks such as Migrate, Revert, Garbage Clean, BLOB Transfer and EBS to RBS Convert make the use of job throttling features. This allows you to fully control the bandwidth consumption in all types of schedules. You can completely stop the bandwidth consumption in peak hours and can slow it down in normal day activity. You can increase the speed or the bandwidth consumption when it is off time or at nights and during weekends.

Hence StorageEdge ensures that WFEs continue to perform at their best when it comes to servicing the user requests at peak load hours, while managing the completion of important tasks through intelligent time-based allocation of precious resources.

Download StorageEdge | StorageEdge Details

Posted in SharePoint enhancement, SharePoint Job Throttling, SharePoint Job Throttling techniques | Tagged , , | Leave a comment

HTML Payload and SharePoint Performance

Posted on September 21, 2011 by Sameer Shah

SharePoint web interface is essentially an ASP.NET application running on WFE servers. As is true for any other web application that larger the HTML payload sent to the browser, greater the amount of time it takes to travel over the wire and finally show. HTML payload depends upon the number of its constituent elements. For instance, like any other ASP.NET application, the size of View State may exceed tens of KBs owing to a large number of controls and widgets, thus becoming a performance bottleneck. Similarly, the extra overhead is introduced by multiple CSS and JavaScript files. So if this footprint could somehow be reduced, it will surely have a considerable impact on the overall SharePoint performance. Let us see how. 

  • View State is sent to the user’s browser to preserve this state across post-backs. For forms with lots of controls, View State can become quite large. The larger the ViewState size is the slower would be the response time for end user between each post back.
  • Similarly, CSS and JavaScript files are major contributors to the SharePoint HTML payload. Whenever a page is rendered, the browser issues separate GET calls for each JavaScript and CSS file. This can easily result into many extra calls to IIS and reduce SharePoint performance.

Optimize SharePoint Performance and Storage - Free Download

So how do we get performance boost? In-memory caching of BLOB payload  tells that caching might be the answer. Why not cache View State after all it is just another object. StorageEdge provides this out-of-the-box solution whereby it caches View State on WFE servers and sends a much smaller payload to the user’s browser containing only a unique ID for this View State. As a result, your page performance improves and SharePoint also scales much better. A snippet of resulting HTML is shown below:

 <input id="__NCPVIEWSTATE" type="hidden" name="__NCPVIEWSTATE"
    value="vs:cf8c8d3927ad4c1a84da7f891bb89185" />
 <input id="__VIEWSTATE" type="hidden" name="__VIEWSTATE" />

Figure 1: StorageEdge Caches View State and Sends an Identifier to the Client

Notice how the original “__VIEWSTATE” hidden field is preserved so everything works as if there was no View State. But, it has inserted its own “__NCPVIEWSTATE” field that it will read when a post-back request comes from the user to the web server. StorageEdge uses the “value” as the key to fetch the corresponding original View State from the in-memory cache and serves it to the ASP.NET page so it can populate the web form with data from the View State.

 Similarly, in case of CSS and Java Script, how about if you could reduce the number of files that form part of HTTP request. StorageEdge employs an intelligent technique called Minification which is a process in which StorageEdge reduces the size of all CSS and JavaScript files and merges them into one or two files. As a result, the web browser only makes one or two extra calls to IIS when requesting a SharePoint page. Thus, you can effectively reduce the number of HTTP requests a browser makes, reduce overall SharePoint HTML payload, and ultimately boost response time.

 So, StorageEdge is not simply an EBS/RBS provider or a BLOB management solution, it is a complete, end-to-end SharePoint management solution. Features such as the ones discussed above take care of all your SharePoint performance needs.

Download StorageEdge | StorageEdge Details

Posted in SharePoint HTML Payload, SharePoint Performance | Tagged , | Leave a comment

Structure your Externalized BLOBs using Multi-Tiered Storage

Posted on August 30, 2011 by Sameer Shah

It is an established industry observation that the data loses its value as it gets older. A fair estimate reveals that in a typical environment, the active data constitutes only 2-4%, aging data is close to 10% and the rest is rarely used. So it is cruel to treat all the data the same in terms of its storage. Same is true for SharePoint where the active BLOB content is only a small chunk of the total content that it carries.

Moreover, everyone is in agreement that there is visible cost saving associated with simply offloading BLOBs from transactional SQL Server storage to less-expensive external storage. However, externalization of BLOBs is only one important aspect of BLOB storage management. How you manage the physical storage of externalized BLOBs is the other one. Keeping all the active and aged BLOBs on one, expensive storage is not a wise idea. If you keep all those type of BLOBs, whether active or worn, in your primary storage tier of external storage, there is a fair chance you will get only a marginal benefit out of externalization, as the cost of storage at your primary, high-end tier shall keep on increasing. So, even after offloading 90-95% of data to external storage, you still manage to get only marginal cost savings, rethink of your storage strategy is in order.

Optimize SharePoint Performance and Storage - Free Download

A worthwhile approach for the storage of externalized BLOBs is structure the external storage as a hierarchy of multiple tiers with one storage tier corresponding to one age-based category of BLOBs. This hierarchical multi tiered storage needs to be structured in such a way that the most active BLOBs should reside at Tier-1 which is a high-end, faster-access storage. Similarly, the aging data should be kept at Tier-2 which is typically a SAN/NAS based storage and the archived/seldom-accessed data at Tier-3 which can typically be a Cloud. So based on this strategy, you effectively push lesser active content to the cheaper tiers.

StorageEdge has been built keeping this very important BLOB storage strategy in perspective. It provides “Multi-tiered Storage” that allows you to keep your active content in the most expensive storage and archives older content out to less expensive storage. This effectively ensures that the primary storage is not over-burdened with millions of documents.

“Multi-tiered Storage” allows you to keep your active content in the most expensive storage and archives older content out to less expensive storage.

Figure 1: Configure separate storage profile for each tier in multi tiered storage.

StorageEdge manages the storage of externalized BLOBs on multi storage tiers through archiving based on two criteria:

1. Age: You can specify content age that allows your content to stay in a tier before it gets migrated to the next tier. This age can be specified separately for each storage tier and you can have as many tiers as you want.

2. Versioning: The other criterion is based on document versioning. Usually, everybody works on the latest copy of the documents so it is practical to archive the older versions to the cheaper storage tier

For Cloud Storage, you may want to control bandwidth, for which StorageEdge provides throttling.

So having multi-tiered storage not only reduces storage costs, but it also improves SharePoint performance because the most active storage no longer contains a huge amount of documents that can potentially inundate it.

Download StorageEdge | StorageEdge Details

Posted in BLOB Externalization, Multi-Tier Storage, SharePoint Storage | Tagged , , , | 1 Comment