Windows Azure has matured nicely over the past few years into a very developer-friendly "Infrastructure-As-A-Service" platform. While many of the recent public announcements have been focused Azure Websites, Deployment from Source Control, and the new general availability of Azure Virtual Machines, one of the core features of the Azure platform remains storage.
Specifically, Azure Blob Storage.
If you are not familiar with Azure blob storage, visit the Azure site, or check out an overview of what it's all about.
Blobs are Great and All That, but What About Files and Folders?
So far as Azure itself is concerned, a blob represents one or blocks of binary data. Much like data on your local hard drive, the notion of directories, files, and the tree-like hierarchal model so familiar to most users is imposed upon the stored data as an abstraction. In the case of your local hard drive, this model is implemented by the operating system.
Within the Windows Azure storage model, there is no OS to impose such structure. Organization and interpretation of the data structure is left up to the client. It is in this way that we are able to store any type of data on the Blob storage platform, and similarly why blob storage is easily consumed by multiple languages and useable from any platform – the structure of the data is platform agnostic.
It's All in the Name
The blob storage model and associated APIs establish addressing and naming conventions which serve to identify individual blobs within storage containers. We address blobs via URLs according to the following format:
http://<your-storage-account-name>/blob.core.windows.net/<container-name>/blob-name
In the above, the URL is composed of your Azure Storage Account name, the /blob.core.windows.net/ namespace, followed by the name of the container in which the blob is located, and finally the name of the blob itself. However, as you recall from our overview, blob names can contain any characters, including forward slashes. For example, one could create a blob named Documents/Photos/Graduation Pic.jpg, which, assuming the following details, would be addressed as follows:
Storage Account: mystorageaccount
Container Name: mycontainer
Blob Name: Documents/Photos/Graduation Pic.jpg
The Blob name above implies that within the container mystorageaccount, there is a directory named Documents, containing a subdirectory named Photos, in which a file named Graduation Pic.jpg is located.
Access this blob using:
http://mystorageaccount/blob.core.windows.net/mycontainer/Documents/Photos/Graduation Pic.jpg
In the Azure storage Account view, a virtual hierarchal directory structure might look like this:
In the image above, note the names of the blobs in the 'Name" column. As far as Azure is concerned, these are simply the blob names. However, much like your operating system, the various SDK implementations afford functionality which can optionally parse these names into a directory structure.
In the above, for example, the blobs File 1.txt, File 2.txt, and File 3.txt can be thought of a being at the root level of the Storage Container, while the blobs named Johns Files/File 1.txt, Johns Files/File 2.txt, and Johns Files/File 3.txt can be thought of as being contained in a directory named Johns Files/. Similarly, We can also see that there is an implied subdirectory within the Johns Files/ directory named Music/, again containing three files.
If we scroll to the right in the Azure browser view above, we see that the same structure is mirrored in the URL for each file, except the file name now conforms to legal URL naming standards:
While Azure storage itself does not recognize the notion of a directory as implied by the delimited blob name, the Azure SDK for your chosen language does, to a certain extent, and assists us in parsing an implied directory structure out of the blob address + name. In this post we will look at the .NET API, however, similar implementations are available for Java, Ruby, Python, and other common platforms.
API Access to the Storage Account and Containers
First we will look at some of the basics involved in accessing an Azure storage account and the containers hosted therein. Using the Azure .NET API, we can create a simple Console application which presents required credentials to the Azure storage account, and then retrieves an IEnumerable <CloudBlobContainer> representing the blob storage containers as follows (note that the various other SDKs and APIs define similar functionality for the appropriate platform/language). To do this, we use the ListContainers() method of the CloudBlobClient class:
// in a real project, one might want to implement some security here:
const string ACCOUNT_NAME = "xiv";
const string ACCOUNT_KEY = "myAccountKey";
const string AZURE_CONNECTION_STRING = ""
+ DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}";
static void Main(string[] args)
{
var storageAccount = CloudStorageAccount.Parse(string.Format(AZURE_CONNECTION_STRING,
ACCOUNT_NAME, ACCOUNT_KEY));
var client = storageAccount.CreateCloudBlobClient();
foreach (var container in client.ListContainers())
{
Console.WriteLine("Container: " + container.Name);
}
// This pauses execution so the console output can be viewed:
Console.Read();
}
In the above, the code simply iterates over the Enumerable and prints the name for each container to the console output. However, the CloudBlobContainer class exposes a variety of useful methods designed to facilitate blob data access. For the purpose of this post, we are primarily interested in the ListBlobs() method.
The ListBlobs method, and the IListBlobItem Interface
The ListBlobs() method returns an instance of IEnumerable<IListBlobItem> which is essentially a list of the blobs within the container. However, the enumerable also includes representations of the implied directory structure.
Items returned by the Instance of Enumerable will one of two types, both of which implement the interface IListBlobItem. Actual blob data ("files" so to speak) will be returned as instances of CloudBlockBlob. Directories, which are essentially abstractions created by the API based upon the structure implied by blob names delimited with forward-slashes, are returned as instances of CloudBlobDirectory.
The instances of IListBlobItem returned in the enumerable can be cast to either CloudBlobDirectory or CloudBlockBlob. From there, we can access properties of each in order to print representations to the console window. For example, if we want to focus on directories only, we could add a method as follows, which iterates over the enumerable passed as an argument, checks the underlying type of each IListBlobItem, and if the underlying type is CloudBlobDirectory, prints the directory prefix to the console window. We then employ a little recursion and do the same for each CloudBlobDirectory instance:
Define the printCloudDirectories Method:
static void printCloudDirectories(IEnumerable<IListBlobItem> blobList)
{
foreach (var blobitem in blobList)
{
if (blobitem is CloudBlobDirectory)
{
var directory = blobitem as CloudBlobDirectory;
Console.WriteLine(directory.Prefix);
printCloudDirectories(directory.ListBlobs());
}
}
}
Now we can add a line to our Main method as defined in the previous section, and after printing the name of each container, we can pass the output of the ListBlobs() method to our new printCloudDirectories method:
Call printCloudDirectories from Main:
static void Main(string[] args)
{
var storageAccount = CloudStorageAccount.Parse(string.Format(AZURE_CONNECTION_STRING,
ACCOUNT_NAME, ACCOUNT_KEY));
var client = storageAccount.CreateCloudBlobClient();
foreach (var container in client.ListContainers())
{
Console.WriteLine("Container: " + container.Name);
// ADD A CALL TO printCloudDirectories:
Program.printCloudDirectories(container.ListBlobs());
}
// This pauses execution so the console output can be viewed:
Console.Read();
}
The output from our little application, if run against the storage account example above, at this point looks like this:
A Simple Console Demonstration
Let's take this a few steps further, and create an application which iterates over the containers of a storage account, and writes a text representation of the implied directory structure to the console output.
In our Main method, we will authenticate as before by presenting our credentials. We will then get a reference to an instance of CloudBlobClient. From here, though, we will make a few minor modifications. We define a method named writeAzureDirectoriesToConsole which accepts an argument of IEnumerable<CloudBlobContainer>. Similar to our previous method, this iterates over the Enumerable, and writes the container name to the console output. From here, however, the output of the ListBlobs() method is passed as an argument to a method named getContainerDirectories, which returns a string representation of the file structure we are after.Unlike our previous example, the getContainerDirectories method employs a LINQ query to parse directories and blob files separately, and also adds some rudimentary indentation to the output for readability. The indented formatting also suggests the tree-like structure we associate with a hierarchal data structure:
// in a real project, one might want to implement some security here:
const string ACCOUNT_NAME = "xiv";
const string ACCOUNT_KEY = "MyAccountKey";
const string AZURE_CONNECTION_STRING = ""
+ "DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}";
static void Main(string[] args)
{
var storageAccount =
CloudStorageAccount.Parse(string.Format(AZURE_CONNECTION_STRING, ACCOUNT_NAME,
ACCOUNT_KEY));
var client = storageAccount.CreateCloudBlobClient();
writeAzureDirectoriesToConsole(client.ListContainers());
// This pauses execution so the console output can be viewed:
Console.Read();
}
static void writeAzureDirectoriesToConsole(IEnumerable<CloudBlobContainer> containers)
{
foreach (var container in containers)
{
string indent = "";
Console.WriteLine("Container: " + container.Name);
// Pass Ienumerable to recursive function to get "subdirectories":
Console.WriteLine(getContainerDirectories(container.ListBlobs(), indent));
}
}
static string getContainerDirectories(IEnumerable<IListBlobItem> blobList, string indent)
{
// Indent each item in the output for the current subdirectory:
indent = indent + " ";
StringBuilder sb = new StringBuilder("");
// First list all the actual FILES within the current blob list. No recursion needed:
foreach (var item in blobList.Where((blobItem, type) => blobItem is CloudBlockBlob))
{
var blobFile = item as CloudBlockBlob;
sb.AppendLine(indent + blobFile.Name);
}
// List all additional subdirectories in the current directory, and call recursively:
foreach (var item in blobList.Where((blobItem, type) => blobItem is CloudBlobDirectory))
{
var directory = item as CloudBlobDirectory;
sb.AppendLine(indent + directory.Prefix.ToUpper());
// Call this method recursively to retrieve subdirectories within the current:
sb.AppendLine(getContainerDirectories(directory.ListBlobs(), indent));
}
return sb.ToString();
}
As we can see, once we have successfully authenticated using our storage account name and associated account key, we create an instance of CloubBlobClient using the CreateCloudBlobClient() method of our StorageAccount instance. From there, we pass an IEnumerable<CloudBlobContainer> (from the ListContainers() method of our client object) to our hacked-together method writeAzureDirectoriesToConsole.
The writeAzureDirectoriesToConsole method simply iterates over the containers passed in, writes the name of the container to the console (in uppercase) and passes each container reference to the slightly more complex getContainerDirectories method, along with an empty string as an initial indentation level. The indentation will help visually display our directory structure in the console by indenting subdirectories and their contents.
The getContainerDirectories method adds four spaces to the indent parameter passed in, then uses a LINQ query to return an IEnumerable<IListBlobItem> for which the instances of IListBlobItem can all be cast to type CloudBlockBlob. each instance is so cast, and then written to the StringBuilder instance for console output after pre-pending the indent.
Next, we retrieve an IEnumerable<IListBlobItem> for which each instance can be cast to type CloudBlobDirectory. As we have seen, the "directories" obtained this way do not represent actual files, but rather an API interpretation of the blob names. Again, each "directory" prefix is pre-pended with the indentation, and then appended to the StringBuilder for output to the console. However, at this point, the getContainerDirectories method is then called recursively, and passed the IEnumerable<IListBlobItem> returned from the current directory's ListBlobs() method, and the result also appended to the StringBuilder for eventual output.
If we run our example against the cloud storage account pictured above, the console output looks something like this:
Simplify the Structure for Display:
We can go a step better here by trimming off the redundant sub-directory paths before we display our tree structure. In the following, I added some lines of code to the which do just that:
static string getContainerDirectories(IEnumerable<IListBlobItem> blobList, string indent)
{
// Indent each item in the output for the current subdirectory:
indent = indent + " ";
StringBuilder sb = new StringBuilder("");
// First list all the actual files within the current blob list. No recursion needed:
foreach (var item in blobList.Where((blobItem, type) => blobItem is CloudBlockBlob))
{
var blobFile = item as CloudBlockBlob;
// If the current blob item is at the root of a container, there
// will be no parent directory. Use the file name as-is:
string outputFileName = blobFile.Name;
// Otherwise, remove the parent directory prefix from the displayed name:
if (blobFile.Parent != null)
{
outputFileName = blobFile.Name.Replace(blobFile.Parent.Prefix, "");
}
// append to the output:
sb.AppendLine(indent + outputFileName);
}
// List all additional subdirectories in the current directory, and call recursively:
foreach (var item in blobList.Where((blobItem, type) => blobItem is CloudBlobDirectory))
{
var directory = item as CloudBlobDirectory;
string directoryName = directory.Prefix;
if(directory.Parent != null)
{
directoryName = directoryName.Replace(directory.Parent.Prefix, "");
}
sb.AppendLine(indent + directoryName.ToUpper());
// Call this method recursively to retrieve subdirectories within the current:
sb.AppendLine(getContainerDirectories(directory.ListBlobs(), indent));
}
return sb.ToString();
}
In the code above, we have simply added some logic to replace the parent directory prefix in the displayed names of subdirectories, using the Parent property of both CloudBlobDirectory and CloudBlockBlob. In both cases, the parent property returns the CloudBlobDirectory which is the parent of the current item. If there is no parent (in other words, if the current item is at the container root level) then the Parent property will return null, so we need to test for this.
After making the above modifications, when we run our code now the console output is a little more readable:
But . . . What Good Is It?
So far, we have taken a very cursory look at the manner in which Windows Azure allows us to impose a directory structure on our blob storage. We can see how we might expand upon these and other aspects of the blob storage API to create a fairly advanced file management application (for example, a desktop or web client) to facilitate management of data, and/or create a stand-alone "cloud drive" type service. As an example, the screenshot below is an early version of a desktop client I am putting together for an upcoming post. Note that it is currently pointed at the same Azure storage account as our console example. Look for this in an upcoming post.
Example of a More Developed Use Case:
The various APIs/SDKs offer a host of properties and methods allowing us to take advantage of Azure Blob Storage features. With the ever-growing cloud-based nature of our daily computer usage, and the nearly ubiquitous, pervasive presence of the internet in our daily lives, the ability for our applications to integrate with scalable, high-availability storage is not going away soon. Services such as Windows Azure, Amazon Web S3, and other API-based cloud storage services will become more and more important in our application architectures.
I will be exploring this topic further in the near future. I hope to shortly have a post discussing implementation of a very basic desktop client application similar to that pictured above. Additionally, I hope to have a similar example of a web-based client which implements at least some basic but effective authorization.
Additional Resources:
John on Google CodeProject
In the course of the past two years Microsoft has made significant strides in creating a developer-friendly experience for using Windows Azure. From a rather disjointed (and to me, poorly understood) beginning, Windows Azure has grown into a fascinating playground for those of us wanting to explore aspects of "Infrastructure-As-A-Service" (IAAS).
At the time of this writing there has been a lot of attention focused upon such sexiness as deploying websites to Azure from integrated source control, and integration of Azure with Microsoft's Webmatrix platform. However, cloud storage is one of the core elements of Azure's IAAS offerings, and will likely play a role in any other Azure services you use.
Azure Blob storage, like Amazon S3, offers a handy (and cheap) way to persist content and make it available across the web. For example, there was a minor kerfluffle last year when Github decided to eliminate the "Downloads" feature of their project hosting platform. Services such as Amazon S3 or Azure Blob Storage present an alternative location to host project binary files and/or other resources for easy linking and download.
Or, if you're like me, you might just find it fun (and informative) to experiment with modeling up your own cloud-based storage client for whatever purpose. On the whole, storage such as provided by Amazon or Azure is incredibly cheap, and it is possible to store a boatload of files for pennies each month (literally).
What Can it Do for Me?
According to the Azure Website, Azure Blob Storage is " . . . a service for storing large amounts of unstructured data that can be accessed from anywhere in the world via HTTP or HTTPS." From the same site, common uses for Azure Blob storage might include (but are definitely no limited to):
- Serving images or documents directly to a browser
- Storing files for distributed access
- Streaming video and audio
- Performing secure backup and disaster recovery
- Storing data for analysis by an on-premises or Windows Azure-hosted service
While the Azure team appears to place the focus on "large amounts of unstructured data," blob storage is equally useful for storing data of any size. As previously mentioned, it is possible to create a basic "cloud drive" type application using Azure storage, or Amazon S3 (I believe Dropbox uses Amazon as its backend).
In the most general sense, the term "blob" is commonly understood to mean "Binary Large OBject." Many of us are familiar with this term from its usage in database-land, where "blob data" might be data stored in our database which does not conform to an established data type as defined by the database. Such data are usually (if the database supports it) persisted as plain binary data (image files come to mind as an example).
Most of the major players in the "cloud" storage space have extended this notion to various generic storage implementations which allow client data of any sort to be uploaded/persisted on the vendor storage server in binary format. Amazon S3 implements a model in which binary data ("Objects") are persisted in "Buckets." Windows Azure persists binary data ("Blobs") in "Containers."
Of course, there is more to it than that, and that's what I am discussing in this post.
An Azure Storage Account will consist of one or more Containers, which are created and named by the user to hold Blobs. All blobs must be located in a container. In general (and at the time of this writing), an Azure user can have up to five separate storage accounts.
An individual storage account may contain an unlimited number of containers, and an individual container may hold an unlimited number of blobs. However, the total size of all containers may not exceed 100TB. Windows Azure defines two distinct types of blob – Block Blobs and Page Blobs. For the moment, I am going to focus on Block Blobs.
According to the Azure team, the most common use-cases for blob storage will involve Block Blobs. Block blobs represent binary data that has been segmented into one or more blocks to enable ease of transmission over a network, and sensible management of large data files. The blocks which make up a blob may be of different sizes, up to 4 MB each. Each block within a blob is identified by a Block ID, and may optionally also include an MD5 hash of the blob content. The maximum size for a block blob is 200 GB, and a blob can consist of up to 50,000 individual blocks.
The idea here is that a large file may be broken up into blocks, which then may be uploaded or downloaded separately, in any sequence (and importantly, asynchronously) and then re-associated with each other, in the proper sequence. The blocks which make up a blob are associated with that specific blob through a list enumerating all the blocks within the blob.
When blocks are uploaded, they are associated with the designated blob, but do not formally become part of the blob until the blocks are Committed, by supplying the list of blocks. Each block identified in the list by its Block ID is formally made a part of the blob. Blocks which are uploaded but not formally committed remain uncommitted, and are discarded after one week. Uncommitted blocks may be committed at any point prior to being discarded. When a client attempts to write a block for a blob that does not exist, a new blob will be created.
Blob data can be modified at the block level. That is, individual blocks can be added to an existing blob, existing blocks can be overwritten or replaced, and specific blocks within a blob can be deleted.
Individual blocks are identified by a Block ID property, which are represented by strings of equal length. The MS Azure site indicates that most blob storage clients utilize base-64 encoding to create ID strings of equal length. Id's must be unique within each blob, but do not need to be unique between different blobs. In other words, blocks in two different blobs may have the same block ID.
Block blob storage is designed to facilitate efficient network file handling and asynchronous file transfer. While the previous may seem a little complex, the Azure team has created API's for a number of popular development languages which make much of this easier to deal with for common use cases.
Container Names must begin with a letter or a number, can contain only letters, numbers, and the dash (-) symbol, and letters must be all lowercase. Container names must be a minimum of 3 and not more than 63 characters in length.
Blob Names can contain any characters, but special characters and characters reserved for use in URLs must be escaped. Blob names can include upper and lower-case letters, and cannot exceed 1024 characters in length.
Container names and blob names are combined to form part of the URL by which the blob is accessed, so names for each must conform to DNS naming standards when combined.
Blobs are accessible through URLs in the following format:
http://<your-storage-account-name>/blob.core.windows.net/<container-name>/blob-name
The Azure team has created a number of SDKs/APIs for programmatically working with blob storage. As of this writing, the following languages/platforms are supported with SDKs/APIs which expose various Azure services, including Blob Storage:
In addition, Azure defines a REST API for all storage services useable by any application which can send/receive HTTP or HTTPS requests/responses. Some of the SDKs above rely on the Rest API for programmatic access.
Additional Resources
John on Google CodeProject
John, Step Away from the Code Editor!
I am a died-in-the-wool nerd. I love my computers, and I love learning to code. Left to my own devices, I will spend the better part of the day and night chopping away at the keyboard, trying to make something cool, or at least something that doesn’t suck. In this, there is danger, though.
Like most (I think) people who tend towards technical pursuits, I have an almost ADD addiction to the work/suffer/reward cycle that comes from busting my ass trying to get the machine to do what I want. Note, I say "do what I want," because, as we know, the machine ALWAYS does exactly "what I say" to do, but that is not always what I "want."
It is so easy for me to get up on a Saturday morning, crack open the internet and the code editor, and before I know it, it's dark outside, and Saturday is gone. Then, do the same thing Sunday, and then start the work week again. I learn a lot. I "get things done" at work. I meet deadlines, and have experienced some modest professional success.
. . . learn to take some time each day to recognize how precious are the moments we have on this planet, and to appreciate that, for one more day, I woke up this morning and drew breath. For one more day, I have received the gift of being allowed to experience this world, and everything it has to offer.
What is it all Worth, Really?
When all is said and done, though, what is all that really worth? When I leave this planet, I can't take any of it with me. The code we write is a nebulous, ephemeral thing which, at best, will be obsolete in a few years. Make no mistake – I recognize the importance of achievement, of securing financial stability for myself and those I care about, and to some degree, attaining sufficient material wealth to live comfortably and with minimal anxiety. But beyond that, what really matters?
. . . the most important things in life are the experiences we have . . . if you do it right, the most important stuff, in this context, is free . . .
About a decade ago, I was forced to take a long, hard look at the way I lived my life, and to begin a long, slow process of realizing what matters, and what doesn't. The story behind that turning point in my life is outside the scope of this blog. But starting then, I learned to take some time each day to recognize how precious are the moments we have on this planet, and to appreciate that, for one more day, I woke up this morning and drew breath. For one more day, I have received the gift of being allowed to experience this world, and everything it has to offer.
Most importantly, to recognize that the most important things in life are the experiences we have, and that if you do it right, the most important stuff, in this context, is free. If we remember to go take it.
A walk through the neighborhood on a warm spring evening . . .
![IMG_0097_thumb[6]](http://typecastexception.com/image.axd?picture=IMG_0097_thumb%5B6%5D_thumb.jpg "IMG_0097_thumb[6]")
There is nothing like the rebirth of Spring, the smells in the air, and the temperate breeze. Often, we get in such a hurry, we forget to get out and ambulate through our environment, our neighborhood, the place we live.
Each evening, I try to get out and walk, and while walking, remain hyper-aware of the everyday, commonplace things around me that I might otherwise take for granted, or miss entirely.
The number of times one gets to experience Spring in this life is finite. Try not to miss the little things.
![IMG_0117_thumb[6]](http://typecastexception.com/image.axd?picture=IMG_0117_thumb%5B6%5D_thumb.jpg "IMG_0117_thumb[6]")
Teaching Someone how to Play the Guitar . . . or do it Yourself, Live . . .
"Music, whether played yourself or absorbed through active listening, has the power to change your psychic state . . ."
![IMAG0202_thumb[2]](http://typecastexception.com/image.axd?picture=IMAG0202_thumb%5B2%5D_thumb_1.jpg "IMAG0202_thumb[2]")
All of my life, music has held sway over me. My mother likes to tell the story of me at age 3, sitting on the floor next to the old turntable with my ear pressed against the speaker, listening intently to the massive snare and cello at the end of the Simon and Garfunkel classic "The Boxer." In my early teens, I learned to play the guitar, and since then, music has been my solace, passion, and escape. In my twenties, I found the sheer, addicting exhilaration of playing, at high volume, to large appreciative audiences. There is truly nothing like it.
![DSCN1641_thumb[1]](http://typecastexception.com/image.axd?picture=DSCN1641_thumb%5B1%5D_thumb.jpg "DSCN1641_thumb[1]")
Now, in middle age, I find the utmost reward in teaching the step-children to play, and in passing well-loved and well-played instruments on to a new generation. My much-loved black Fender Stratocaster (seen in the hands of the youngest step-child, above) was handed down last year (to the same step daughter. No worries – I replaced it with a new pearl white one!).
Music, whether played yourself or absorbed through active listening, has the power to change your psychic state. Passing that passion on to another, and sharing the impact music can have on your life is priceless.
. . . The Exhilaration of the Breeze off the Pacific Ocean
"Standing on the Westernmost edge of the American continent, gazing out across the largest ocean in the world, it is easy to remember how small we really are in the big scheme of things . . ."
![Cropped Haystack Rock_thumb[6]](http://typecastexception.com/image.axd?picture=Cropped%20Haystack%20Rock_thumb%5B6%5D_thumb.jpg "Cropped Haystack Rock_thumb[6]")
![DSC03877_thumb[1]](http://typecastexception.com/image.axd?picture=DSC03877_thumb%5B1%5D_thumb.jpg "DSC03877_thumb[1]")
![DSC03892_thumb[2]](http://typecastexception.com/image.axd?picture=DSC03892_thumb%5B2%5D_thumb_1.jpg "DSC03892_thumb[2]")
I have been to three coasts in my lifetime (so far), but none creates for me the same sense of peace as the Northwest Pacific Coast. Standing on the Westernmost edge of the American continent, gazing out across the largest ocean in the world, it is easy to remember how small we really are in the big scheme of things. Important to keep in mind, as it is so very easy to mentally overstate our own importance, and the seeming importance of the petty events which make up our daily lives.
Of course, it's also a great place to spend time with friends and family.
Do you have a sea coast? A place where you can feel small, and recognize that is the way it is supposed to be?
Fall at Concordia Seminary in the St. Louis Suburbs . . .
![Natural Impressionism_thumb[10]](http://typecastexception.com/image.axd?picture=Natural%20Impressionism_thumb%5B10%5D_thumb.jpg "Natural Impressionism_thumb[10]")
Fall is a universal season. But growing up in the American Pacific Northwest, I had never experienced the exceptional explosion of color such as I saw when I moved to St. Louis, Missouri.
The days remained temperate (sometimes, even warm), but a chill was in the air as the deciduous trees on the grounds of Concordia Seminary here in Clayton (a suburb, in a way, of St. Louis) performed their annual magic show.
It was at this point I recognized that, while Portland, Oregon, will always be "home," there is a natural beauty to every part of the world. It's all a matter of looking from the right perspective.
Here in the middle of the country, I feel far from my home, friends, and family. But I get to experience the fall in a way I never have before.
And that's a treat.
. . . A Sunset at the Gulf Coast of Alabama in Mid-August . . .
![DSCN2655_thumb[3]](http://typecastexception.com/image.axd?picture=DSCN2655_thumb%5B3%5D_thumb.jpg "DSCN2655_thumb[3]")
In the Summer of 2012, I was afforded the opportunity to visit the Gulf Coast of Alabama for a week. After a fascinating drive through the deepest parts of Mississippi, we arrived. The Gulf is unlike the Pacific beaches back home. The heat and near-tropical humidity gave an entirely different feel. The oil rigs operating offshore added a synthetic enhancement to the skyline. And as the sun sets through the moist, humid southern air, the sky takes on a fascinating palette of pastel colors, while a warm breeze picks up.
The culture along the gulf coast, while definitely American, also had its own flavor. And you haven't had gumbo until you've had it prepared at a mom-and-pop restaurant here in the place of its origin.
The Monuments We make . . .
![Arch Extreme Oblique_thumb[3]](http://typecastexception.com/image.axd?picture=Arch%20Extreme%20Oblique_thumb%5B3%5D_thumb.jpg "Arch Extreme Oblique_thumb[3]")
For me, just as it is important to remember how small mankind is in the big scheme of things, it is equally important to remember that we are capable of great victories, despite the laws of nature, and despite seemingly impossible odds.
How DOES one construct a 600 + foot high catenary arch of stainless steel?
While the monuments our species has created over the millennia might be seen as a grand delusion of our own self-importance, they can also serve to remind us as individuals that we are part of a greater whole, That whole, small though it may be on the universal scale of things, is greater than the sum of its parts, and capable of great things.
We all leave this planet with exactly what we brought into it – nothing, and alone. But when the last of us is gone, there will remain, for a time, some evidence we were here, and that we conquered some of the most basic forces of nature and subjugated them to our will, more so than any creature which inhabited this planet before us.
The People in our Life
It’s really easy to get caught up in the frantic here and now, in chasing professional success, material or monetary gain, or other ephemeral "things." I doubt anyone has ever said, on their deathbed, "If only I had worked more".
No one will ever accuse me of being a "people person." However, as I grow older, I find that the people in my life, from various times and places, have made it what it is today.
Studies of how people experience happiness have shown that, while acquiring shiny new things provides a transitory jolt of seeming satisfaction, it is not lasting, and the individual moves on to seek the next thing. On the other hand, shared experiences, such as camping with friends, vacations with family, and some of the things I describe throughout this post are the type of thing which create lasting memories, and ultimately, a full life.
To quote something I found on the inter-webs recently, Life is a one-time offer. Best get it right. TO me that means making the most of the time we have with the people in our lives. Often, the people in our lives are complete strangers whom we have not taken the opportunity to know better yet.
When you leave this world, would the story of your life make a great movie? To quote the philosopher Morrison:
“Is everybody in? Is everybody in? Is everybody in? The ceremony is about to begin. The entertainment for this evening is not new, you've seen this entertainment through and through you have seen your birth, your life, your death....you may recall all the rest. Did you have a good world when you died? -enough to base a movie on??”
- Jim Morrison, An American Prayer
John on Google
TL:DR – For a newcomer to Linux, likely starting out with Ubuntu or Linux Mint, installing applications that are not part of the Software Manager or Synaptic Package Manager catalog for the chosen distro is not always intuitive at first. I take a rather long look at how to get Sublime Text 2 properly installed on your machine, and do my best to explain what is going on, rather than simply provide terminal entries to copy and paste. I would be happy to hear from experience Linux users about how I might improve, or where I have explained something poorly.
A few months back, I decided to expand my horizons and explore Ruby and Rails. I also decided that I would do so in the more native Linux environment, rather than go the Ruby-on-Windows route. This was one of the best decisions I have made in terms of developing my skills and experience as a programmer.
The learning continues. I started with Linux Mint 13, which has a friendly enough GUI, but for most of what I am doing, I try to use the Bash CLI as much as possible. I've never been very comfortable with the command line, and so long as I am learning a new language, in a new OS environment, I figured it was time to overcome that limitation as well.
If you are an experienced Linux user, there is probably nothing here for you. This is really basic, and yet I had to look around and cull some information from a variety of sources in order to figure out how to do this.
Why Sublime Text 2
Unless you have been in a coma, you have no doubt at least heard of Sublime Text 2. Sublime Text 2 is a cross-platform (Mac/Windows/Linux), highly customizable text editor with an outstanding compliment of features. I have found it to be my preferred, go-to tool for text manipulation. While it is not an IDE, Sublime Text presents sufficient IDE-like features that for many purposes, you won't miss the extra cruft. Also, of course, in many programming communities (Ruby, JavaScript), code is more often written in a text editor.
NOTE: As of this writing, Sublime Text 3 is available to registered Sublime Text users as a beta release.
Sublime Text 2 is not currently part of the Synaptic Package Management system on Linux Mint (or Ubuntu). Therefore, there is no magical apt-get install command as you might use to install other software on your Linux system, so we have to do a little more work.
Of course, the straightforward method of installing Sublime Text 2 on your Linux Box is to download the appropriate (23 or 64-bit) .tar file from the Sublime Text site, unpack, and locate in the directory of your choice. You can do this manually by going to the Sublime Text 2 Downloads page and clicking the appropriate download link, or you can do it all from the terminal, as described below.
If you are not as familiar with Bash command line as you would like, see my previous posts. While these were written in the context of using Git for Windows developers, the basic Bash commands are explained:
This method is described on the Sublime Text Site/Support/Linux/Installation page. Simply open a terminal in the directory you use for applications, and enter the following command (use the appropriate version fro your machine):
NOTE: As of this writing, Sublime Text 2.0.1 is the most recent stable release. If the stable release is updated, the URL's in the links below will change, and you will need to copy the updated URL from the Sublime Text site.
Download the Linux 32-Bit Version of Sublime Text 2:
$ wget http://c758482.r82.cf2.rackcdn.com/Sublime%20Text%202.0.1.tar.bz2
Download the Linux 64-Bit Version of Sublime Text 2:
$ wget http://c758482.r82.cf2.rackcdn.com/Sublime%20Text%202.0.1%20x64.tar.bz2
Extract the "Sublime Text 2.0.1.tar.bz2" file (this will be "Sublime Text 2.0.1 x64.tar.bz2" for the 64 bit version):
Extract the Sublime Text .tar file:
tar vxjf "Sublime Text 2.0.1.tar.bz2"
Then you can add a sym link to the executable file with a short name for convenience (it seems to be a convention to use the alias "subl" for ease of use from the terminal. The executable file will be located in the extracted Sublime Text 2 directory. For example, if you extracted the .tar contents into a directory ~/apps then the sublime_text executable will be "home/Sublime Text 2/sublime_text" (since there are spaces in the directory name, we need to use quotes around the path).
Add a Sym link:
sudo ln -s "~apps/Sublime Text 2/sublime_text" /usr/bin/subl
The above method is easiest, but does not leave you with a convenient way to update Sublime Text in the future short of removing the current installation, re-downloading, and re-installing. There is an arguably better method, which relies on the Personal Package Archive system.
Canonical, the company which supports Ubuntu, has created the Launchpad.net site which, among other things, hosts a repository for Personal Package Archives (PPA's). Here, individuals and teams can upload their own software and installation packages, and it is possible to find deployment packages for software that is not included in the Ubuntu or Linux Mint Synaptic Package Manager for your specific distribution. It is also possible to add the PPA to your Synaptic catalog, so that you can then run apt-get install, apt-get update and the like to keep your package up to date.
Or, at least as up to date as the package maintainer at Launchpad keeps theirs.
The WebUpd8team at Launchpad has created (among other things) a PPA for Sublime Text 2 which is up to date with version 2.0.1 as of this writing. To add Sublime Text 2 to your Synaptic catalog, and install according to the install script published with the PPA, follow these steps:
Add the Sublime Text 2 Repository to your Synaptic Package Manager:
sudo add-apt-repository ppa:webupd8team/sublime-text-2
Update:
sudo apt-get update
Install Sublime Text:
sudo apt-get install sublime-text
Next, check the usr/bin directory. You should see at least one file, named sublime-text-2, and you should also see two others, named sublime-text and subl. These create aliases you can use to invoke Sublime Text 2 from the command line. If the subl and sublime-text files are not present, copy the sublime-text-2 file and make them:
Create alias files (if not present):
$ sudo cp /usr/bin/sublime-text-2 /usr/bin/sublime-text
$ sudo cp /usr/bin/sublime-text-2 /usr/bin/subl
There you have it. You can now use Sublime Text 2 from you command line. Also, you should see it available in your GUI in the applications menu.
This has been a long post about a relatively simple operation. My goal has been to explain the concepts as fully as possible, under the assumption that there are those out there, like myself, new enough to Linux to need the extra handholding. Thanks for reading!
John on Google CodeProject
TL;DR -
Webmatrix 3 is cooler than you might think. It offers a very convenient set of features which to my mind are a complement to, and not replacement for, a more robust development environment. Particularly for doing quick touch-ups to HTML or CSS for an existing site, tracking such changes in source control, and easily publishing the updates to Azure or your standard hosting provider. The addition of source control integration and Azure deployment make Webmatrix exceptionally useful in addition to your more robust development tools.
Microsoft Webmatrix 3 will never be confused with a full-fledged application development tool. Nonetheless, with the version 3 release of this lightweight, free suite of site creation and management tools, Microsoft has, in my opinion, hit something of a sweet spot in tooling for easily building, deploying, managing and maintaining basic web sites and/or sites created from various template packages.
Let’s face it, these days, if you need a simple blog, CMS, forum, or other tried-and-true site category, odds are good you will not be rolling your own, but instead using (perhaps with some customization) an existing package such as Wordpress, Joomla, Orchard CMS, or some other framework. Or it may be that you are creating a simple static site which does not require a full complement of development tools such as Visual Studio.
Webmatrix 3 provides a handy set of features for maintaining and deploying changes to a site without having to crack open a full solution in Visual Studio. Minor tweaks to HTML or CSS are quick and easy. Also, one of the highlights of this new release is some basic source control integration, notably absent from previous versions. Lastly, built-in deployment to the Windows Azure platform, either directly using Webmatrix, or by hooking your Azure-hosted site up to your project repository such that pushing to branch master automatically publishes changes to your live site.
While in some ways Webmatrix looks a little like a toy, I strongly suggest that you give it a closer look before dismissing it completely. Using Webmatrix 3, it is possible to get a basic site up and running quickly and with little fuss, particularly for those who need little more than to deploy and lightly customize templates available in the extensive Web Application Gallery, included out of the box. Also notable is that, since version 2, Webmatrix supports Node.js an PHP, in addition to the expected ASP.NET Web Pages.
Basic File Management and Code Editing
As I mentioned previously, Webmatrix is not a full IDE. However, it does provide a competent text editor with competent syntax highlighting and autocomplete, with a full complement of dictionaries for HTML and CSS:
Git Source Control Integration
The most attractive feature of the new release, for me, is the source control integration. When you click on the Source Control tab at the top of the editor window, you are presented with a choice to use Git or Microsoft’s own TFS. Personally, I have no use for TFS, so let’s see what the integrated Git offering is all about.
As soon as you click on the Git option, Webmatrix initializes a new Git repository in the local folder in which you are working, and presents you with a few GUI controls offering the commonly used Git functions:
The GUI-based Git functionality provided here is very basic. To perform any but the basic functions identified by the ribbon items available in the window shown above, one must crack open the Shell. One complaint I have is that the “Shell” icon seen in the picture above does not open a standard Bash command shell (which is what I have become accustomed to with Git) but instead a Windows Powershell CLI.
The Webmatrix 3 Powershell Window for Git:
The msysGit/Git for Windows/Bash CLI I Have Come to Know and Love:
Webmatrix Git integration appears to depend on libgit2Sharp, a .NET wrapper for the C-based libgit2 port of native Git functionality into a portable, platform-agnostic library of core Git functionality. While I am not fully-versed on the implications of this, I DO notice that the Git/Bash command line interface, as made available in Webmatrix, is not the same as I am accustomed to, either in msysGit (on Windows) or when using Git on my Linux box. While most things appear to behave in a (mostly) familiar way,I found it disorienting at first (no doubt this is a reflection on my unfamiliarity, and not the implementation). I will be doing some more exploring to see what the differences are here.
On the other hand, if I am needing much more advanced Git/Source Control functionality, no doubt I should probably be moving to a more powerful tool altogether. Another option is to use the standard Git for Windows command line when such advanced git features are needed (provided you have Git for Windows or msysGit installed). Happily, on the “Home” menu tab, there is a “Launch in Visual Studio” button. In other words, when you need more power, it’s there (provided you have Visual Studio installed).
Windows Azure Integration
If you develop for the Microsoft stack (and even if you don’t) it has been difficult to miss the growing presence of the Windows Azure platform, and specifically, Windows Azure Websites. Currently, you can create up to 10 sites on the Windows Azure platform for free. There are some trade-offs, in that under the free model, you can’t use custom domain names, and there are some resource limitations. Additionally, it looks like upgrading to the paid model might quickly become more expensive than traditional shared hosting. However, the Azure platform as a hosting provider shows increasing promise, and some very cool features.
As with most Microsoft products, integration with their Windows Azure platform has become a first-class citizen in Webmatrix 3. From the minute you install the product and run it for the first time, you are presented with opportunities to add your Azure account (if you have one) or create a new one ( if you don’t). Likewise (and more pertinent to this article), there are multiple ways to rather seamlessly take advantage of Azure as a host for sites you create in Webmatrix.
From the moment I first create or open a new website on my local machine, I am able to deploy immediately to my Azure Account:
Open a Local (existing) Website Folder:
Select the Folder Containing Your Website Files:
You will then be asked if you want to create a new Azure Website (provided you have your Azure account set up):
Create Azure Site Dialog:
Once you click “OK” your existing site will be provisioned on your Azure account, and publicly visible at the link you see in the yellow notification box at the bottom of the screen.
Site Provisioned on Your Azure Account:
At this point, I can work on my site locally, and test changes by running it in the built-in IIS development server installed with Webmatrix.
Changes Automatically Uploaded When You Publish
What has happened behind the scenes here is Webmatrix has taken care of setting up an Azure website on your account, and all of the required MS Web Deploy in order to publish changes. For example, say I have made a few changes to the CSS or HTML for my site. I can preview them by clicking the “Run” button on the Home tab, which will open the site in my default browser running in my localhost IIS Server:
If I like what I see, I can then use the “Publish” button to automatically upload my changes to the live Azure site.
Other Deployment Options
Of course, publishing directly to an Azure website is only one of the simplified deployment options available in Webmatrix 3. Also present are standard Web Deploy and FTP deployment settings so that changes to your site can be easily published to the hosting provider of your choice.
Another Tool in the Chest
Webmatrix 3 is cooler than you might think. It offers a very convenient set of features which to my mind are a complement to, and not replacement for, a more robust development environment. Particularly for doing quick touch-ups to HTML or CSS for an existing site, tracking such changes in source control, and easily publishing the updates to Azure or your standard hosting provider. The addition of source control integration and Azure deployment make Webmatrix exceptionally useful in addition to your more robust development tools.
There is a rich set of basic features for common deployment and site maintenance scenarios available here. While as I have said, Webmatrix is emphatically not a full-fledged development tool, it is a handy addition, particularly for quick and dirty site creation and maintenance, or for those who need a simple, easy to learn way of getting something published on the web.
Additional Resources:
CodeProject John on Google