We achieve stable, near-continuous deployment at Charlie by following a basic development flow. The idea is simple: incrementally improve our product, deploy frequently (several times a day!), and be process-driven to achieve a stable service with a lightning-fast feedback loop.

You’ll achieve

  • Stress-free deployments: Anyone can deploy, at any time
  • Stable codebase: Everything is reviewed by peers and business and tested before it’s merged into master
  • Fast user feedback loop
  • All developers familiar with majority of codebase

We build software similar to how GitHub builds software, but we make a few significant changes that we feel are incredibly important:

  • Test & review changes in the QA environment before merging to master
  • Continuously deploy master to Staging

The Development Flow

  1. Branch Master
  2. Code & write tests
  3. Deploy to a QA environment
    1. Open a Pull Request
    2. Peer code review (unless you pair)
    3. Business review & discussion (as needed)
    4. Merge pull request
  4. Continuous deployment to Staging
  5. Smoke test your changes
  6. Deploy to Production

The Rules

  • Master is always stable & deployable
  • Deploy frequently, incremental changes (several times a day!)
  • Anyone can deploy to production
  • Thoroughly test your code (unit / integration / acceptance)
  • Automated deployment script
  • Continuous Integration on all remote branches

1. Branch Master

Create a branch from master for all changes you make (features and bug fixes). Give it a friendly name, like “company-news” or “fix-twitter-date-error.”

2. Write Code / Write Tests

Regardless of how you get there, adequately test your code. This includes a healthy mix of unit tests, integration tests, and acceptance tests. Ensure you take into consideration the different scenarios and use cases for your features. That’s all I’ll say about that because this is a topic beyond the scope of this post. I’m not going to ignite a TDD flame war, so watch this series for that debate: Is TDD Dead?

3. Deploy to QA

Here’s the missing key to the GitHub flow.

When you’ve wrapped up development, deploy your branch to QA for review and testing. The QA environment represents production plus your changes. The main goal here is to catch any issues and ensure things are implemented as expected before the code is integrated with master.  Otherwise, you’d have master in a non-deployable state.

As a developer, you’ll uncover deployment issues before it hits master. For some reason every time you go from local to a hosted environment, you discover issues. May as well find then before they pollute the codebase!

Run a single command to deploy to QA:

ey deploy -e qa

Our QA environment is completely isolated from other environments, and is similar to production, but not identical. We have one web server, one Sidekiq worker, a separate MongoDB, and Redis. We have four developers on our team and one QA environment suffices. For a larger team, you may want to have additional QA environments.

Open a Pull Request

Open a Pull Request to start the conversation. Document your changes or simply link to the card in Trello. If necessary, list out instructions for how to test or reproduce, and include anything noteworthy like design decisions. Include screenshots of your new functionality from your QA environment using business-relevant data. Make it dead simple for the reviewer to see what has changed by annotating the screenshots if necessary (we use Skitch to do that).

Your Pull Requests become an amazingly detailed change log, complete with a writeup, screenshots, and every line of code that’s changed, with discussion.

Peer Code Review

github-pull-request

Every line of code changed is reviewed by a fellow team member. Any team member, except for new team members, performs code reviews. If you pair program, you have automatic code reviews built in, so this step isn’t necessary. It’s important that you have everyone on the team review code so that:

  1. All team members are familiar with the entire codebase
  2. Junior team members learn from how more senior members code
  3. No bottlenecks if you have only the senior members code review

For new team members and especially new junior developers, have a senior team member pair up and review the code with them to educate them on how to properly review code. Do this until you feel they’re ready to fly.

Business Review

In QA, the business folks can try out your new functionality before it’s integrated into master and give you fast feedback (all documented in the pull request). Sure, you’re UX person created beautiful mockups, but sometimes when you actually play with the feature, it just doesn’t feel right. It’s much easier to make those tweaks before the developer has shipped the feature and moved onto something else. Once it’s live, you’re more likely to work on something else and push off those minor tweaks until never.

Merge Your Pull Request

Once business has signed off (only when necessary), code review checks out, and the build passes, it’s ready to ship. The developer who opened the pull request closes the pull request. This is important because it allows the developer to put on any finishing touches and keeps her in control. Of course, if you make significant changes, ask your developer for a quick follow-on review of the final commit. Delete your branch locally and off the server once merged (don’t worry, Github always keeps a copy).

4. Verify in Staging with Continuous Deployment

Your staging environment represents what’s about to go to production. It’s not too far off, because it’s only ahead by one or a few pull requests at the most. The staging server configuration represents a very close (but scaled down) version of production. reference an article The build server continuously deploys master to staging after all tests have passed.

Verify in Staging

Before you deploy to production, do a sanity check in staging. Poke around with the new functionality and make sure everything’s as it should be. The real goal is to make sure that the integrated changes will work as expected in production. You want to catch any deployment issues before they’re live. You already have tests in place to ensure existing functionality remains in-tact, so you don’t need to go crazy and test features that are already live. This is just a precaution, but it’s a really good idea, especially for larger changes.

We keep track of what’s sitting in staging in a separate list on our Trello board, so you know what you’re about to deploy.

trello

5. Deploy to Production

Here’s the fun part. No more stress-induced, ceremonial deployments. When it’s time to go live, any developer on the team runs a simple command that automatically deploys the code from Github. Trust the process and feel confident that your deployment will go fine.

“Software release can–and should–be a low-risk, frequent, cheap, rapid, and predictable process.”
- Martin Fowler, Continuous Delivery

Here’s our deployment command:

ey deploy -e production

Deploy during the day

We have our environment configured for zero downtime, so we can deploy multiple times a day, during the day. It’s important that you deploy during the day as opposed to at night. Why? During the day everyone’s sharp and on-point. There’s also time to recover if things go wrong. If you deploy at night, you may go to sleep and wake up to down alerts. That’s no fun!

Conclusion

We build Charlie following this development flow.  We’ve had zero downtime in the last 6 months, largely due to this process.  It’s a simple process that uses multiple environments to deliver software, where each is more stable than the last (Local > QA > Staging > Production).  The important thing here is every small change is peer code-reviewed and reviewed by business (as needed) before it’s integrated into master.  We use Continuous Integration to ensure all of our tests (we have thousands) pass before deploying any change.  It’s an amazing feeling to deploy to production and not feel in the least bit anxious about something going wrong.

What do you think?

I’d love to hear your feedback.  What do you think about our process?  How do you deliver software?

 

 

References

GitHub Flow, Scott Chacon, 8/31/2011

How GitHub Users GitHub to Build GitHub (video), Zach Holman, 9/21/2011

Continuous DeliveryMartin Fowler, 8/2010

bruce_on_rails

Who says installing Ruby on Rails on a Windows box is a pain in the ass?  It isn’t.  You can be a Rails developer and rock a Windows machine too.  It’s easy to configure and install Rails on Windows, if you know where to look.

Update:

At first, running Rails on Windows worked out pretty well.  It was a bit slow, but that’s all.  Then I tried using Sidekiq to kick off background jobs.  That depends on Redis (an intelligent in-memory object store), which isn’t available on Windows without an unofficial port by Microsoft.  Many technologies are simply just not fit for Windows and so I needed a new solution.

Vagrant to the rescue!  Vagrant allows you to script portable VM development environments.  It’s amazing – new developers need to run a simple command and within minutes, they have a fully configured development environment ready to go.  You can edit and debug locally and run the VM server in the background.  Don’t bother trying to run Rails on Windows.  Just use a VM via Vagrant.  It’s so money.

Run Rails on a Vagrant VM

I configured a VM to make it easy for new developers on my team at Charlie to start coding in only a few minutes.  Try it out: it comes with the latest and greatest Rails 4.0, Ruby 2.0, MongoDB, and Redis.  It’s awesome and I know you’ll love it.

Download our Rails VirtualBox

 

Install Ruby on Rails on Windows (if you must)

Follow the steps….

  1. Rails windows installer (all in one installer goodness)
    railsinstaller
    http://railsinstaller.org 

  2. Install Sublime Text - your simple code editor
    http://www.sublimetext.com
  3. Configure Sublime Text according to this blog post
  4. Download Anonymous Pro - fixed-width font to add a lil style to you environment
  5. Get Tabbed Command Prompts with ComEmu
    Scott Hanselman loves it too!
  6. Create your shell Rails project & pick your Ruby gems
    http://railswizard.org
    Run the command they tell you from c:\dev
Posted in Dev.

How do you design for relational data in a non-relational database, i.e. NoSQL database, like Amazon’s DynamoDB or mongoDB?  Pretty much all data has some sort of relationship or connection to other data, so how do we model this in a non-relational database?

You have a couple of options.

Here are some solutions that I thought of whole designing the db structure on my new startup, Route Scout.

  1. Mock a relational db design (not good.)
    You could use a typical relational design by storing an array of ID’s in the parent table that references records in another table.  Since you don’t have access to join operators when querying the table, you have to make multiple read queries to get back all of the data.  You may even have to make one query for every record if your db doesn’t support some kind of IN operator (Dynamodb doesn’t).  This is slow and costly, so lets think of a better solution.
  2. De-normalize your data (gasp!)
    Replicate a subset of the data from the child table in the parent table.  This makes reads very quick and is the way to go.  Read below…

De-normalization Design Pattern

In our de-normalized database, we’ll store some data from a child table in each related table.  We only store the data that we need to query, so our database doesn’t get too bloated.  When you query the parent table, you’ll get back all of the related data because it’s stored with each record in the parent table.  One query gets you all of the data you need.  And it’s super fast because the database isn’t making any joins.

You need to think about the different queries you’ll need to run.  The types of queries you’ll need will direct the design of your database, so make sure you thoroughly think about the different use cases when you start designing your database.  Let’s go through an example:

Example

Say we need two tables: Users and Trips.  Users can plan zero or more trips.  This is a simplified version of the actual database used by Route Scout.

Use Cases

  1. List a user’s trips (display the trip destination name and photo)
  2. Display a trip, along with the user who planned it (we only need the user’s ID, name, and photo)
  3. Create or edit a trip

DB Schema

From the use cases, we design a simple schema as shown below:

No-SQL DB blog post (3)

We’ve duplicated some of the User data in the Trips table, and some of the Trip data in the Users table.  I call the subset of duplicated data “summary” data.  This allows us to make very simple queries and get all of the data we need.   Using DynamoDB, I store the summary data in JSON format.

Actual Data

Here’s an example of the actual data stored in DynamoDB.  Notice how the trip & user summary data is stored as a JSON string.  Nice and compact!
Trip DB table

routescout-data

Fast Reads, Slow Writes

Reads are extremely fast because you’re typically making only one db query to get back a record and the related data that you need.  Writes on the other hand, are slow.  When you create or edit a child record, you need to update that record in all places where it’s stored.

Update all duplicated data in parallel

You can execute these multiple write queries in parallel to save time.  Note that if you’re on DynamoDB, you’ll need a high write capacity to execute these calls at the same time.

With this approach, the write methods in your business layer become a bit more complex than in a traditional relational database.  Unit testing very critical here to ensure data is updated in all places in which it’s stored.

Example

Update trip destination title to “Breckenridge, CO” for a trip with 3 travelers, user A, B, and C, execute each of the following in parallel:

  • Update trip record
    trip:
    { ID: "123", Destination: { Title: "Breckenridge, CO" }, User: { ID: "user-A" }, Travelers: [ { ID: "user-B" }, { ID: "user-C" } ] }
  • Update trip summary on user A
    user:
    { ID: "user-A", Trips: { Title: "Breckenridge, CO" } }
  • Update trip summary on user B
    user: 
    { ID: "user-B", Trips: { Title: "Breckenridge, CO" } }
  • Update trip summary on user B
    user: 
    { ID: "user-C", Trips: { Title: "Breckenridge, CO" } }

Conclusion

Working with a NoSQL database like Amazon’s DynamoDB or MongoDB can be tricky, especially coming from a traditional relational database background like SQL Server or MySQL.  The benefits of a NoSQL db are clear in my opinion: easy to scale horizontally, cheap to operate with high-availability even at low-volume using a cloud-hosted service, and flexible schemas (each row can have different fields).  It takes a bit of patience to learn the nuances and best practices of a NoSQL database, but it’s worth it for sure.  With proper planning and thought when designing your db structure, you’ll end up with a database that’s easy to work with and very quick.  Make sure you unit test all of the methods that interact with the database!

Have a question or comment?  I’d love to hear your feedback.

Follow me @bobbychaz

Like what you read?  Share this post with your friends below.

When developing a web app using the Facebook Javascript SDK, you may want to attach multiple event listeners to the Facebook load event.  If you have the code to load and initialize Facebook’s SDK in a common include, then it’s tricky have separate pages handle the load event differently.  Or maybe you have one page and want to handle the load event multiple times in different areas of code.  The solution is simple.

Here’s How

All you have to do is fire a custom event using jQuery when the Facebook SDK loads, like so:

// fire a custom jQuery event
$(document).trigger('fb-load');

Here’s the complete example

<div id="fb-root"></div>
<script type="text/javascript">// <![CDATA[
window.fbAsyncInit = function() {
    // init the FB JS SDK
    FB.init({
      appId      : 'YOUR_APP_ID', // App ID from the App Dashboard
      channelUrl : '//WWW.YOUR_DOMAIN.COM/channel.html', // Channel File for x-domain communication
      status     : true, // check the login status upon init?
      cookie     : true, // set sessions cookies to allow your server to access the session?
      xfbml      : true  // parse XFBML tags on this page?
    });

    // fire a custom jQuery event
    $(document).trigger('fb-load');
  };

  // Load the SDK's source Asynchronously
  // Note that the debug version is being actively developed and might 
  // contain some type checks that are overly strict. 
  // Please report such bugs using the bugs tool.
  (function(d, debug){
     var js, id = 'facebook-jssdk', ref = d.getElementsByTagName('script')[0];
     if (d.getElementById(id)) {return;}
     js = d.createElement('script'); js.id = id; js.async = true;
     js.src = "//connect.facebook.net/en_US/all" + (debug ? "/debug" : "") + ".js";
     ref.parentNode.insertBefore(js, ref);
   }(document, /*debug*/ false));
// ]]></script>

Now you can handle the Facebook Load event multiple times!

$(function () {
    $(document).bind('fb-load', function () {
        // do something
    });
});
$(function () {
    $(document).bind('fb-load', function () {
        // do something else
    });
});

Here’s a quick MVC HTML extension method that outputs a JS date object.  It’d be nice if JS had a way to represent a date literal outright, but this works just the same.  The tricky part with the Date() constructor is the inconsistent parameters.  Year is the year, Month is the month 0-11, Day is the day of month (1-31), and the times are all zero-based.

Usage

<script type="text/javascript">
var d = <%: Html.JSDate(DateTime.Now) %>;
</script>

Output

// today is 8/22/2010
var d = new Date(2010,7,22,19,7,34,532)

Source

///
/// Outputs a javascript date object declaration
///
public static string JSDate(this HtmlHelper html, DateTime date)
{
    return string.Format(“new Date({0},{1},{2},{3},{4},{5},{6})”,
        date.Year, date.Month – 1, date.Day, date.Hour, date.Minute,
        date.Second, date.Millisecond);
}

My client, a super-huge pharmaceutical company, gave me a particularly good challenge: take a string of HTML and truncate it to a maximum amount of characters, keeping only whole words and all markup.  The trick of course, is to close any open tags, not cut off the text in the middle of a tag, and not to leave a partial word at the end.  I whipped out my trusty pen and paper and started drawing out the algorithm.  It turned out to be pretty straightforward, but there’s a lot to watch out for.

The Basic Idea

  1. Count the raw character count and the text count until you hit the maximum number of text characters
  2. Truncate the string to the character count, keeping whole words
  3. Create a Stack to hold all open tags
  4. Use some nifty regex to identify HTML tags
  5. Loop through and push all tags to the Stack and pop them off as we find closing tags
    1. Try to recover if we have malformed HTML and find tags closed in the wrong order
  6. Pop any remaining tags off the stack and add them as closing tags to our truncated string

You can download the source here, along with some other nice HTML String Extensions like Truncate() and TruncateWords().  I’ve included some unit tests as well.

Source Code (with a few bonus extension methods)

using System.Collections.Generic;
using System.Text;
using System.Text.RegularExpressions;

namespace System
{
    public static class StringHtmlExtensions
    {
        /// <summary>
        /// Truncates a string containing HTML to a number of text characters, keeping whole words.
        /// The result contains HTML and any tags left open are closed.
        /// </summary>
        /// <param name="s"></param>
        /// <returns></returns>
        public static string TruncateHtml(this string html, int maxCharacters, string trailingText)
        {
            if (string.IsNullOrEmpty(html))
                return html;

            // find the spot to truncate
            // count the text characters and ignore tags
            var textCount = 0;
            var charCount = 0;
            var ignore = false;
            foreach (char c in html)
            {
                charCount++;
                if (c == '<')
                    ignore = true;
                else if (!ignore)
                    textCount++;

                if (c == '>')
                    ignore = false;

                // stop once we hit the limit
                if (textCount >= maxCharacters)
                    break;
            }

            // Truncate the html and keep whole words only
            var trunc = new StringBuilder(html.TruncateWords(charCount));

            // keep track of open tags and close any tags left open
            var tags = new Stack<string>();
            var matches = Regex.Matches(trunc.ToString(),
                @"<((?<tag>[^\s/>]+)|/(?<closeTag>[^\s>]+)).*?(?<selfClose>/)?\s*>",
                RegexOptions.IgnoreCase | RegexOptions.Compiled | RegexOptions.Multiline);

            foreach (Match match in matches)
            {
                if (match.Success)
                {
                    var tag = match.Groups["tag"].Value;
                    var closeTag = match.Groups["closeTag"].Value;

                    // push to stack if open tag and ignore it if it is self-closing, i.e. <br />
                    if (!string.IsNullOrEmpty(tag) && string.IsNullOrEmpty(match.Groups["selfClose"].Value))
                        tags.Push(tag);

                    // pop from stack if close tag
                    else if (!string.IsNullOrEmpty(closeTag))
                    {
                        // pop the tag to close it.. find the matching opening tag
                        // ignore any unclosed tags
                        while (tags.Pop() != closeTag && tags.Count > 0)
                        { }
                    }
                }
            }

            if (html.Length > charCount)
                // add the trailing text
                trunc.Append(trailingText);

            // pop the rest off the stack to close remainder of tags
            while (tags.Count > 0)
            {
                trunc.Append("</");
                trunc.Append(tags.Pop());
                trunc.Append('>');
            }

            return trunc.ToString();
        }

        /// <summary>
        /// Truncates a string containing HTML to a number of text characters, keeping whole words.
        /// The result contains HTML and any tags left open are closed.
        /// </summary>
        /// <param name="s"></param>
        /// <returns></returns>
        public static string TruncateHtml(this string html, int maxCharacters)
        {
            return html.TruncateHtml(maxCharacters, null);
        }

        /// <summary>
        /// Strips all HTML tags from a string
        /// </summary>
        /// <param name="s"></param>
        /// <returns></returns>
        public static string StripHtml(this string html)
        {
            if (string.IsNullOrEmpty(html))
                return html;

            return Regex.Replace(html, @"<(.|\n)*?>", string.Empty);
        }

        /// <summary>
        /// Truncates text to a number of characters
        /// </summary>
        /// <param name="text"></param>
        /// <param name="maxCharacters"></param>
        /// <param name="trailingText"></param>
        /// <returns></returns>
        public static string Truncate(this string text, int maxCharacters)
        {
            return text.Truncate(maxCharacters, null);
        }

        /// <summary>
        /// Truncates text to a number of characters and adds trailing text, i.e. elipses, to the end
        /// </summary>
        /// <param name="text"></param>
        /// <param name="maxCharacters"></param>
        /// <param name="trailingText"></param>
        /// <returns></returns>
        public static string Truncate(this string text, int maxCharacters, string trailingText)
        {
            if (string.IsNullOrEmpty(text) || maxCharacters <= 0 || text.Length <= maxCharacters)
                return text;
            else
                return text.Substring(0, maxCharacters) + trailingText;
        }


        /// <summary>
        /// Truncates text and discars any partial words left at the end
        /// </summary>
        /// <param name="text"></param>
        /// <param name="maxCharacters"></param>
        /// <param name="trailingText"></param>
        /// <returns></returns>
        public static string TruncateWords(this string text, int maxCharacters)
        {
            return text.TruncateWords(maxCharacters, null);
        }

        /// <summary>
        /// Truncates text and discars any partial words left at the end
        /// </summary>
        /// <param name="text"></param>
        /// <param name="maxCharacters"></param>
        /// <param name="trailingText"></param>
        /// <returns></returns>
        public static string TruncateWords(this string text, int maxCharacters, string trailingText)
        {
            if (string.IsNullOrEmpty(text) || maxCharacters <= 0 || text.Length <= maxCharacters)
                return text;

            // trunctate the text, then remove the partial word at the end
            return Regex.Replace(text.Truncate(maxCharacters),
                @"\s+[^\s]+$", string.Empty, RegexOptions.IgnoreCase | RegexOptions.Compiled) + trailingText;
        }
    }
}

Unit Tests

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Helpers.Net.Tests
{
    /// <summary>
    /// Summary description for StringHtmlExtensionTests
    /// </summary>
    [TestClass]
    public class StringHtmlExtensionTests
    {
        public StringHtmlExtensionTests()
        {
            //
            // TODO: Add constructor logic here
            //
        }

        private TestContext testContextInstance;

        /// <summary>
        ///Gets or sets the test context which provides
        ///information about and functionality for the current test run.
        ///</summary>
        public TestContext TestContext
        {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;
            }
        }

        #region Additional test attributes
        //
        // You can use the following additional attributes as you write your tests:
        //
        // Use ClassInitialize to run code before running the first test in the class
        // [ClassInitialize()]
        // public static void MyClassInitialize(TestContext testContext) { }
        //
        // Use ClassCleanup to run code after all tests in a class have run
        // [ClassCleanup()]
        // public static void MyClassCleanup() { }
        //
        // Use TestInitialize to run code before running each test 
        // [TestInitialize()]
        // public void MyTestInitialize() { }
        //
        // Use TestCleanup to run code after each test has run
        // [TestCleanup()]
        // public void MyTestCleanup() { }
        //
        #endregion

        [TestMethod]
        public void ToDelimitedStringTest()
        {
            Assert.AreEqual("", (new string[] { }).ToDelimitedString(","));
            Assert.AreEqual("", ((string[])null).ToDelimitedString(","));
            Assert.AreEqual("one", (new string[] { "one" }).ToDelimitedString(", "));
            Assert.AreEqual("one, two", (new string[] { "one", "two" }).ToDelimitedString(", "));
            Assert.AreEqual("one,two", (new string[] { "one", "two" }).ToDelimitedString(","));
        }

        [TestMethod]
        public void StripHtmlTest()
        {
            Assert.IsNull(((string)null).StripHtml());

            Assert.AreEqual("hello", "hello".StripHtml());

            Assert.AreEqual("hello", "he<b>ll</b>o".StripHtml());
        }

        [TestMethod]
        public void TruncateTextTest()
        {
            Assert.IsNull(((string)null).StripHtml());

            string test = "1234567890";
            Assert.AreEqual("12345", test.Truncate(5, null));
            Assert.AreEqual("12345...", test.Truncate(5, "..."));
            Assert.AreEqual(string.Empty, string.Empty.Truncate(5, null));
            Assert.AreEqual("12", "12".Truncate(5));
        }

        [TestMethod]
        public void TruncateHtmlEmptyTest()
        {
            Assert.IsNull(((string)null).TruncateHtml(100));
            Assert.AreEqual(string.Empty.TruncateHtml(100), string.Empty);
        }

        [TestMethod]
        public void TruncateHtmlTextTest()
        {
            // no html test
            Assert.AreEqual("abc".TruncateHtml(10), "abc");
            Assert.AreEqual("abc".TruncateHtml(2), "ab");
        }

        [TestMethod]
        public void TruncateHtmlTest()
        {
            var html = @"<p>aaa <b>bbb</b>
ccc<br> ddd</p>";

            Assert.AreEqual(@"<p>aaa <b>bbb</b>
ccc<br> ddd</p>", html.TruncateHtml(100, "no trailing text")); // it ignores unclosed tags

            Assert.AreEqual(@"<p>aaa <b>bbb</b>
ccc<br>...</br></p>", html.TruncateHtml(14, "..."));

            Assert.AreEqual(@"<p>aaa <b>bbb</b></p>", html.TruncateHtml(10));

            // self closing test
            html = @"<p>hello<br/>there</p>";
            Assert.AreEqual(@"<p>hello<br/>th</p>", html.TruncateHtml(7));

            Assert.AreEqual("<b>i'm</b>", "<b>i'm awesome</b>".TruncateHtml(8));
            Assert.AreEqual("<b>i'm...</b>", "<b>i'm awesome</b>".TruncateHtml(8, "..."));
        }

        [TestMethod]
        public void TruncateWordsTest()
        {
            Assert.IsNull(((string)null).TruncateWords(100));
            Assert.AreEqual(string.Empty, string.Empty.TruncateWords(100));

            Assert.AreEqual("big brown", "big brown beaver".TruncateWords(12));
            Assert.AreEqual("big...", "big brown beaver".TruncateWords(5, "..."));
        }

        [TestMethod]
        public void TruncateWordsBreakingHtmlTagTest()
        {
            // truncates in the middle of a tag
            Assert.AreEqual("<b>i'm", "<b>i'm awesome</b>".TruncateWords(16));
        }
    }
}

Are you interested in seeing a real-world architecture of an application on Windows Azure?  I wrote an article that details the architecture of my side project, an aggregate RSS Feed Reader called Feed Me Daily.  It shows how you can use Azure to scale your application as your audience grows.  The example architecture uses multiple web and worker roles, AppFabric Caching for its distributed caching layer, and Azure Table Storage  for its database.

Check out the Architecture of Feed Me Daily

This is one of those ambiguous SharePoint 2007 COM+ exceptions that kept us troubleshooting at 2am. Many different causes and symptoms produce an HRESULT error. The one I’m covering here is HRESULT 0×80020009 (DISP_E_EXCEPTION). The particular symptom that we see in this case is when we try to add a custom web part on a page that read data from pages in the Pages Library. We check in a draft of the page then add an out of the box webpart like the content editor. Either when we add the content editor, or after we try to check in any changes, we get the very nasty HRESULT Exception.

In order to get some more detail on the cause of this ambiguous SharePoint HRESULT exception, we need to turn on verbose logging, reproduce the error, then dive into the SharePoint error logs. In our case, we got the following GetFileFromUrl: ArgumentException:

10/15/2009 00:34:19.42 w3wp.exe (0×10630) 0x84C4 CMS Publishing 6wyd Medium GetFileFromUrl: ArgumentException when attempting get file Url http://stage.yoursite.com/_catalogs/masterpage/YourPageLayout.aspx Value does not fall within the expected range. at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetMetadataForUrl(String relUrl, Int32 mondoProcHint, Guid& listId, Int32& itemId, Int32& typeOfObject, Object& fileOrFolder) at Microsoft.SharePoint.SPWeb.GetFileOrFolderObject(String strUrl) at Mic…

Notice the page layout URL points to staging. The site in question is in production. So the issue here is a custom webpart is reading data from a page that references a page layout from a different environment.

This happens sometimes when you manually move pages across environments using SharePoint Designer, i.e. from staging to production.

Solution

The good news is it’s easy to spot and fix this incorrect page layout referencing error.

  • In SharePoint Designer, click Task Panes -> Hyperlinks
  • In the Hyperlink pane, click the bottom “Show Internal Hyperlinks” icon (bottom icon on left side)
  • Click the “Verify Hyperlinks” icon and verify all hyperlinks (top icon on left side)
  • Sort the results by the Hyperlink column and scroll to links starting with http://
  • Look for any links referencing page layouts in an environment other than your current environment, i.e. staging when the page is in prod
  • Right click and go to Edit Hyperlink. Change the reference to the production URL and click Replace
  • Verify the changes by re-running “Verify Hyperlinks”

I really hope this helps for some of you. There are many causes to an HRESULT error in SharePoint and this is one possible solution. Be sure to leave comments below.

Big thanks to Josh Gaffey from our project team for figuring out the solution.

Many times I’ll need to parse the domain name, including the http[s]:// from a URL.  Yet there’s no straightforward way to get it using the Uri class.  This is especially useful when writing custom webparts for SharePoint.  I like to avoid using the Uri class anyway because it tends to just be a headache to use (it doesn’t serialize, you need to check for a null or empty string, object overhead, and what’s-the-point-anyway).  I wrote two C# extension methods that parse a domain from a URL string or a Uri object using Regex.

Useage

“http://bing.com/hello”.AsDomain(); // => “http://bing.com”
“https://bing.com/hello”.AsDomain(); // => “https://bing.com”
“http://bing.com:1234/hello”.AsDomain(); // => “http://bing.com:1234″
“/hello”.AsDomain(); // => “/hello”

I like to write methods that are forgiving, in that they don’t complain by throwing exceptions when inputs aren’t quite right.  You can use the method on an empty string, a null string, a relative domain, or even a string that’s not even a domain.  In those cases, the method just returns the input string.  This way it’s very easy to use and you don’t need to check for IsNullOrEmpty() every time.

Source Code: Parse URL from a string, C# extension method

namespace System
{
    public static class StringExtensions
    {
        /// <summary>
        /// Parses the domain from a URL string or returns the string if no URL was found
        /// </summary>
        /// <param name="url"></param>
        /// <returns></returns>
        public static string AsDomain(this string url)
        {
            if (string.IsNullOrEmpty(url))
                return url;

            var match = Regex.Match(url, @"^http[s]?[:/]+[^/]+");
            if (match.Success)
                return match.Captures[0].Value;
            else
                return url;
        }

        /// <summary>
        /// Parses the domain from a URL
        /// </summary>
        /// <param name="url"></param>
        /// <returns></returns>
        public static string AsDomain(this Uri url)
        {
            if (url == null)
                return null;

            return url.ToString().AsDomain();
        }
    }
}

Complete with Unit Tests

Just so you know it’s been tested at least a small bit.  I combine most tests into a single method because I’m lazy.  Deal with it.

        /// <summary>
        ///A test for AsDomain
        ///</summary>
        [TestMethod()]
        public void AsDomainTest1()
        {
            Assert.IsNull(((string)null).AsDomain());
            Assert.AreEqual(string.Empty, string.Empty.AsDomain());
            Assert.AreEqual("http://www.bing.com", "http://www.bing.com/hello".AsDomain());
            Assert.AreEqual("http://localhost:1234", "http://localhost:1234/hello".AsDomain());
            Assert.AreEqual("http://www.bing.com", "http://www.bing.com".AsDomain());
            Assert.AreEqual("https://www.bing.com", "https://www.bing.com/hello".AsDomain());

            Assert.AreEqual("/relative", "/relative".AsDomain());
        }

        /// <summary>
        ///A test for AsDomain
        ///</summary>
        [TestMethod()]
        public void AsDomainTest()
        {
            Assert.AreEqual("http://bing.com", new Uri("http://bing.com/hello").AsDomain());
        }

I used this amazing online app called RegExr to build the regular expression used here.  It’s a great replacement for Expresso (my trial ran out!).  Check it out, it’s awesome.