Tuesday, October 13, 2009

Forcing the SharePoint object model to reload the 12 hive

Or: A bit of insight into the inner workings of the SharePoint object model

This isn't necessarily something many have found themselves battling, but I most certainly did yesterday. As I was trying to deploy a feature containing a site definition, and subsequently create a site using the site definition, using my new SPDeploy application, it all abruptly came to a nasty halt as I added the site (using SPSiteCollection.Add()):

File or arguments not valid for site template 'SomeTemplate#0'

Having triple-checked the template, feature, solution and just about everything else (yes, SPDeploy also does the equivalent of stsadm -o execadmsvcjobs), it eventually dawned on me how SharePoint goes about things when you create a new site.

Just about any call in SharePoint will go through whatever object model class you're dealing with, such as SPSiteCollection, then to a SPRequestManager class, on to an SPRequest, SPRequestInternalClass and eventually into an arcane COM class (which appears to be part of the old SharePoint 1.0 structure, when COM, ISAPI and such was the shizznit). This COM class exposes hundreds of functions, ranging from list administration to tepmlate lookup.

The issue in my case was two-fold. First of all, any thread in an application which uses the SharePoint object model, will access a set of SPRequest objects (based on whether you need an authenticated or non-authenticated instance) from the SPRequestManager class. These are per-thread request objects, which execute on a first-come, first-served basis. In figuring this out, I began experimenting with releasing the SPRequests for the main thread in the SPDeploy tool - which would cause them to be reinitialized in a subsequent call. I verified that this would not hurt other stateful objects, such as already lookup'd SPWebApplication instances; as all functions in these classes will acquire their SPRequest instance from SPRequestManager. And as I said: the SPRequestManager would create a new instance if an old one wasn't found.

After typing up the reflection madness to clear the current thread's SPRequest objects, I gave the application another go. But still got the error - doh.

This confirmed my worst fears: the SPRequestInternalClass COM object, which is obviously native code, did its own caching. Pulling out Sysinternals' ProcMon and watching loads from the filesystem beginning with "web server extensions\12" backed this up further: web templates, and the rest of the hive, is only loaded once - on demand - then cached.

In realizing this, the solution became pretty clear, and given the nature of SPRequestManager; not too shabby at all. For tools which require adding stuff to the 12 hive (such as deploying features with web templates), and then *use* those templates without re-executing the tool: you have to recycle the SPRequests in SPRequestManager, and recycle the OWSSVR.dll COM library.

This *also* applies for PowerShell scenarios, where you during one session call into one of the SharePoint objects, such as e.g.:
([type]"Microsoft.SharePoint.Administration.SPWebApplication")::Lookup($uri);

Doing so will load the OWSSVR.dll into your current session, and if you happen to e.g. trigger a function from the object model which scans the 12 hive for tepmlates; any subsequent script based addition to this folder, and then attempted usage of the additions, will fail with the error noted above.

The workaround I apply in SPDeploy, which may look intrusive, but is completely harmless in the single-tool scenario, is:

[DllImport("kernel32.dll")]
private static extern IntPtr FreeLibrary(IntPtr library);

[DllImport("kernel32.dll")]
private static extern IntPtr GetModuleHandle(string lpModuleName);

[DllImport("kernel32.dll")]
private static extern IntPtr LoadLibrary(string lpFileName);

public static void RecycleOwssvr()
{
PropertyInfo threadCtx = typeof (SPFarm).GetProperty("ThreadContext", BindingFlags.Static | BindingFlags.NonPublic);
var tbl = (Hashtable) threadCtx.GetValue(null, null);
PropertyInfo requestProp = typeof (SPFarm).GetProperty("RequestNoAuth", BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic);
requestProp.GetValue(SPFarm.Local, null);
tbl.Remove(typeof (SPFarm).Assembly.GetType("Microsoft.SharePoint.SPRequestManager"));

IntPtr p = GetModuleHandle("OWSSVR.DLL");
FreeLibrary(p);
string stsadmPath = SPUtility.GetGenericSetupPath("ISAPI");
p = LoadLibrary(stsadmPath + @"\OWSSVR.DLL");
}

And that will work beautifully.

One note, though; the timer service has to be running for this recycling to work, otherwise some other internal component will freak out come second-go.

Sunday, October 11, 2009

SPDeploy update

CodePlex link: http://spdeploy.codeplex.com

I just uploaded a new binary and source package to the CodePlex site, with some structural, config and visual improvements.

First of all, web application extensions are now supported, meaning that web applications you either lookup or create as part of the configuration can be extended to intranet / extranet / custom zones. A custom rolemanager, membershipprovider and ssl usage can be specified through the attributes. See the xsd schema in the Schemas folder for more info on which attribues are available for the WebApplicationExtension node.

Further, the logging now looks a lot better, thanks to clearer language and some more elaborate code behind the output formatting. I've also refurbished a lot of the general processing code, to make it easier to extend for possible project helpers. I've got more changes coming later, but with a pretty limited amount of time on my hands, I can't be specific as for when.

There are still some hidden general behavior settings being tossed about in the background, which aren't obvious unless you turn to the code. I'll document and clear those up in a future (probably the next) release.

Friday, October 9, 2009

Introducing SPDeploy - a quick way to (re)deploy dev and test structure to SharePoint

CodePlex link: http://spdeploy.codeplex.com

About

SPDeploy will, based on schema validated xml configuration, lookup or create content in a local SharePoint farm, such as WebApplications (with content database and iis site / application pool), Sites, Webs. It will also handle wsp solution installations to the farm; further deployments of installed wsps into web applications; activation of features on all feature-accepting levels of the structure.

Folder, list item and document support is planned, but not as a replacement for WSPs, features or plain common sense.

With SPDeploy you can quickly, and painlessly, reset and redeploy your test or development environment, as well as quickly add development / third party features to the mix - without incorporating them into your production ready wsps, or extending large powershell / batch scripts. The point is to encourage proper integration testing in your development environment, and bringing structure to your favorite SharePoint build server.

Important! Deployment or upgrades into a production or even UAT environments should be made with as strict a setup as possible, with (if possible) all the structure definition, creation and library deployment packaged into definitions, features and solutions. I do not suggest using SPDeploy for this. Test with SPDeploy - ship with wsps and upgradable features!

Developer / extension notes

All data classes in the project are generated from XSD, and generic configration runners make it easy to expand with custom sub nodes. Add an accepted sub node to the XSD, rebuild the project to generate new data classes, and create a Processor derived class to deal with the new sub node type. A new sub node processor will automatically get the result from the lookup or creation of the node above (like a SPWeb, if you're adding processors under the XML's Web node).

Sample xml configuration data

If you reference the schema provided in the source package, Visual Studio will even give you autocompletion and validation of the configuration as you go.


<?xml version="1.0" encoding="utf-8" ?>
<Deployment xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Schemas\SPDeploy.xsd">
<Solutions>
<Solution FileName="C:\path\to\a\Solution.wsp"/>
</Solutions>
<WebApplications>
<WebApplication Action="Lookup" Name="SharePoint - 80">
<SolutionDeployments>
<SolutionDeployment Name="Solution.wsp"/>
</SolutionDeployments>
<Sites>
<Site Action="Create" Url="/" Title="RootWeb" LCID="1044" Template="STS#1" OwnerLogin="somedomain\user">
<RootWeb>
<Features>
<FeatureActivation Guid="fba9e8df-b758-4197-8768-169ef727d5bb" />
</Features>
<Webs>
<Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
<Webs>
<Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
<Webs>
<Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
<Webs>
<Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
</Web>
</Webs>
</Web>
</Webs>
</Web>
</Webs>
</Web>
</Webs>
</RootWeb>
</Site>
</Sites>
</WebApplication>
<WebApplication Action="Create" Name="TestApp" Header="vm-dev" Port="8009" Database="TestContent" PoolName="TestPooly" CreatePool="true">
<SolutionDeployments>
<SolutionDeployment Name="Solution.wsp"/>
</SolutionDeployments>
<Sites>
<Site Action="Create" Url="/" Template="STS#1" LCID="1044" OwnerLogin="somedomain\user" Title="TestWeb">
<RootWeb>
<Webs>
<Web Action="Create" Title="SubWeb" LCID="1044" Template="STS#1" Url="SubWebUrl">
<Features>
<FeatureActivation Guid="fba9e8df-b758-4197-8768-169ef727d5bb" />
</Features>
</Web>
</Webs>
</RootWeb>
</Site>
</Sites>
</WebApplication>
</WebApplications>
</Deployment>

Monday, October 5, 2009

Assigning dynamic ids to HTML and Script elements - Client Side!

This question popped up on Twitter yesterday:

How do you usually handle duplicate IDs for html elements and scripts in SharePoint Web Parts, scripts and controls?

Which is a natural concern, with all the different web parts and scripts that's being injected SharePoint portals these days. To end Monday off (in my neck of the woods anyway), I thought I'd make a quick post on what I do in my projects.

Script naming convention

For scripts, I have a few guidelines. First of all, my scripts follow a naming convention based on my organization handle, library concern, project name and so forth. Looking back to my previous couple of posts, e.g. the Script Loader (http://www.codefornuts.com/2009/10/pieces-of-my-core-javascript-library.html), this name would be grep.scriptloader - as that's a core piece of script for me. For a script specific to a web part, I'd name it similar to the webpart, e.g. grep.webparts.navigationcontrol.

As for the actual script definitions, two matters apply: is it a class, or a single instance?

For single instances, such as the Script Loader above, I declare it like this:

if (!window.grep) window.grep = {};
if (!grep.someSingleInstanceClass)
{
grep.someSingleInstanceClass=
{
instanceVar: "baz",
foo: function() {},
bar: function() {}
}
}

So basically I check if it's already been assigned, and if it hasn't, I assign it to a global variable. The above code would then be accessed such as

grep.someSingleInstanceClass.foo();

For multi instance classes, I follow regular javascript oo syntax, with the same has-it-been-defined-clause, such as:

if (!window.grep) window.grep = {};
if (!grep.someClass)
{
grep.someClass = function(ctorVariable)
{
this.someVar = ctorVariable;
this.DoWhatever();
}

grep.someClass.prototype.DoWhatever = function()
{
// ...
}

grep.someClass.prototype.AnotherFunction = function(input)
{
// ...
}
}

Then access that such as

var foo = new grep.someClass("some value for the ctor");
foo.AnotherFunction(42);

HTML element lookups, and dynamic ids

For elements, I use another trick. Since giving the elements a specific id can easily crash with other web parts of the same kind, or even other web parts / controls, I tend to avoid using ids as much as possible, and rather rely on class lookups (where the class has been named similar to the web part). For such lookups to be possible, however, they must be based somewhere - you can't just search for all instances of 'css class foo' in all of the body tag.

Imagine the following scenario:

<div>This element will be added several times, by other instances of the same web part</div class="webPartInstanceFrame">

<script type="text/javascript">
new grep.webparts.someFrameManager(id of the previous div);
</script>

The script convention noted further up would make sure that the class' code will only be added once, and we feed that the id of the div, so that the class can make css class based jQuery (or similar) lookups to find the elements it needs within the frame. But what's the id of the div? Assigning a static id to it would crash with other instances of the same web part, and using a class would be just as hopeless.

That's where the trick applies, and the following is a fully functional example:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script type="text/javascript">
function renamePrevious(tagType, id)
{
$(document.body.lastChild)
.prev(tagType + ":first")
.attr("id", id);
return id;
}
</script>

<div>I want a dynamic ID!</div>

<script type="text/javascript">
var dynamicId = renamePrevious("div", "SomePrefix" + new Date().valueOf());
/* renamePrevious should be passed as an argument in dynamicId's place, to avoid all variable assignments! */
document.getElementById(dynamicId).innerHTML += " - OK!";
</script>

The trick works by looking up the script block's element in the html dom. This element is the last added element, as the script block executes, even if it's not at the bottom of the page. Based on this script element, it searches its previous siblings for the first match of a given tag type, then assigns a new id.

You could do a lot better with the actual dynamic id than the millisecond postfix shown here, such as an incrementing (or even random) number which is added to a string, then checking if that already exists (in which case we increment / re-random and try again). I leave that to your imagination, though.

As for the example further up, the frame could be identified and put into action such as:

<script type="text/javascript">
new grep.webparts.someFrameManager(
renamePrevious("div", "E" + new Date().valueOf()));
</script>

And that would enable us to avoid any and all name crashes for our component.