Biggy: A Fresh Look at High-Performance, Synchronized In-Memory Persistence for .NET

Posted on March 22 2014 04:58 PM by jatten in C#, Database, Biggy   ||   Comments (2)

the-grid-house-by-elif-ayiter

About a month ago, Rob Conery popped off with the following tweet:

rob-conery-biggy-tweet

And hence, Biggy was born. Rob's "most ridiculous thing he has ever created" has quickly morphed into a high-performance document/relational query tool that is fully LINQ-compliant.

Biggy is an open source project hosted on Github, and is in the early stages of active development. Things have evolved rapidly, and I am proud to say I have contributed some real code and fixes on this, from almost the beginning. This is an exciting project with some interesting, convention-challenging aspects and some cool and really, really knowledgeable leaders and contributors, and I am just damn stoked to be a part of it.

Look, ma! I'm an open source contributor!

Related:

What the Hell is it?


Well, we're not precisely sure . . . ok, we kind of are now. Biggy began as Rob's experiment with a flat-file JSON store, which would be materialized into an in-memory, queryable list structure, which is synchronized with the flat-file store for writes (inserts and updates).

Soon enough, he began experimenting with integrating document-style persistence of data as JSON with standard relational database structures. In Biggy Basics, Part I Rob discusses his thoughts on appropriate persistence models for different types of data:

Our store is a simple process - the "input" data (Products and Customers) generate "output", or "record" data. The input data changes fairly often - Customers logging in and changing things, store owners changing prices, etc.

The output data doesn't change much - in analytical terms this is called "slowly changing over time" - you might go in and tweak an order here and there, but mostly it's a matter of historical record and should never be changed.

To me the "input" stuff (Products/Customers) is perfect for a looser, document structure. The output stuff should not only be in a relational structure - it should be denormalized, ready for analytical export to CSV or some other reporting system.

It was around this time that I jumped in. I've been looking for an OSS project to jump into, for a variety of reasons, and this seemed perfect. The size and scope were such that one could easily grasp the whole of the project. Further, I cut my coding teeth on database stuff, so this was right up my alley.

Ultimately, what Biggy does is combine many of the best characteristics of several data management and persistence models in one package. Biggy is a high-performance, synchronized in-memory query and persistence tool for .NET.

Or, as the README file on Github states, Biggy is "A Very Fast Document/Relational Query Tool with Full LINQ Compliance"

Fast, In-Memory Queries

One of the fundamental aspects of Biggy as a data access tool is that your backing store is represented completely in memory using your domain object model. Where this truly shines is in query performance. Once your data is loaded up, queries are executed, via LINQ, against this in-memory object model.

For example, as things sit currently we could "new up" our data in-memory from the Chinook sample database, and run the following code, with a multi-join LINQ query against it (the Biggy API is evolving as we speak, but this code works against the current master branch available from the Biggy repo as of 3/22/2014):

Example Multi-Join LINQ Query against a Biggy Relational Store:
var _artists = new SQLServerList<Artist>(_connectionStringName, "artist");
var _albums = new SQLServerList<Album>(_connectionStringName, "album");
var _tracks = new SQLServerList<Track>(_connectionStringName, "track");
  
var recordsLoaded = _artists.Count() + _albums.Count() + _tracks.Count();
var actracks = from ar in _artists
             join a in _albums on ar.ArtistId equals a.ArtistId
             join t in _tracks on a.AlbumId equals t.AlbumId
             where ar.Name == "AC/DC"
             select t;
foreach (var track in actracks)
{
    Console.WriteLine("\t-{0}", track.Name);
}

Note, where we "new" up the table data into memory for the first time, we take a little hit as the connection pool is initialized. Not much of one, though! From there, things are FAAASSSST. Here are some really rough performance numbers from my machine (everything is different from one machine to the next):

biggy-acdc-query-perf

Indeed, that triple-join query returned in 1 millisecond. As for the Loading in 117 ms, the bulk of that was the opening of the connection in the first place.

The idea here is that (within sensible parameters determined by our application needs), we can load our application data into memory, and eschew round trips to the database except to perform writes, to keep our in-store model in sync with the back-end.

Structured, Normalized Relational Storage

Let's face it. Despite the advent and popularity of the many flavors of NoSQL, the relational model of data persistence is not going anywhere. Further, Structured Query Language (SQL) also happens to be an excellent way to talk to a database. While often reviled by programmers, I say, man up. When it comes to describing what you want from your data store, SQL beats any "Map-Reduce" style DSL hands down for ease of use and readability.

Also, for a good many types of data, relational data storage is simply (as of this writing, anyway) the best available model, time-tested and mature.

Biggy plays nicely with relational databases. The current implementation focuses on SQL Server and PostgresSql, but as it evolves, it becomes ever easier to port implementation for other SQL database platforms.

As data is added, updated, or deleted in memory, Biggy maintains sync with the backing store without having to reload the in-memory cache.

Loosely Structured, De-normalized Document Storage

For more modest persistence needs (small web applications and the like) Biggy offers an evolved version of Rob's original flat-file JSON store concept. For tables of small to moderate size, performance is strong, and management of data as simple as pointing Biggy at the directory you want to use as your database, and you're off and running. Biggy will de-serialize your JSON data into memory as domain objects. New data is appended to the end of the file as it is added to the in-memory list, and updates are written by flushing all data back to the file.

Biggy also brings the document storage model to your relational database. Domain objects can be persisted as documents within the relational structure, taking advantage of the existing indexing capabilities of the relational Db, and the efficient, flexible record structure of a document Db. Records are saved to a document table indexed with a standard Primary Key, with the actual data serialized as JSON in a "body" field.

This is especially well-suited for PostgreSQL, with its JSON data type. However, SQL Server does just fine with JSON serialized as text. As you might expect, you can save complex objects as a JSON document, and re-serialize later with any parent/child relationships intact:

For example, consider two tables in our database, Artist and Album. We represent these in our domain as you would expect:

Example Artist and Album Classes:
public class Artist 
{
    public int ArtistId { get; set; }
    public string Name { get; set; }
}
  
public class Album 
{
    public int AlbumId { get; set; }
    public string Title { get; set; }
    public int ArtistId { get; set; }
}

 

Using Biggy's hybrid relational/document features, we could decide we need a handy way to store and retrieve artist data in a de-normalized format, such that the artist, and their album catalog, were persisted as a single JSON document, ready for retrieval. We can simply add a handy document container class to our model:

Extending the Artist Class for Document Persistence:
public class ArtistDocument : Artist 
{
    public ArtistDocument() 
    {
        this.Albums = new List<Album>();
    }
    public List<Album> Albums { get; set; }
}

 

Now, we could write something akin to the following code:

var _artists = new SQLServerList<Artist>(_connectionStringName, "artist");
var _albums = new SQLServerList<Album>(_connectionStringName, "album");
  
var list = new List<ArtistDocument>();
foreach(var artist in _artists) 
{
    var artistAlbums = from a in _albums
                     where a.ArtistId == artist.ArtistId
                     select a;
    var newArtistWithAlbums = new ArtistDocument() 
    {
        ArtistId = artist.ArtistId,
        Name = artist.Name,
        Albums = artistAlbums.ToList()
    };
    list.Add(newArtistWithAlbums);
}
var _artistDocuments = new SQLDocumentList<ArtistDocument>(_connectionStringName);
_artistDocuments.AddRange(list);

 

In the code above, we new up our Artist and Album data from the Chinook backing store, and use it to hydrate our document container class, ArtistDocument. We then create a new SQLDocumentList<ArtistDocument> instance.

But wait, John, you say, there's no table for this in Chinook Db. Well, you're correct, there's not. But Biggy knows this, and takes care of that for us. When we initialize a DocumentList<T>, Biggie creates a table for us if one does not already exist.

Once that's done, we just drop our list of complex objects into the AddRange() method and we're done. If we look at our backing store, we see the following table data:

Artist Document Data Persisted in SQL Server as Complex JSON:

complex-artist-documents

The image above may be hard to see in detail, be we have persisted each ArtistDocument record with a serial integer Primary Key, and a body of JSON, which includes the artist data, as well as a JSON array representing the album catalog for each artist.

A Closer look reveals standard JSON:

{"Albums":[
	{"AlbumId":163,"Title":"From The Muddy Banks Of The Wishkah [Live]","ArtistId":110},
	{"AlbumId":164,"Title":"Nevermind","ArtistId":110}],
"ArtistId":110,"Name":"Nirvana"}

 

And yet, in our in-memory list, each artist will be serialized back into its proper domain model, with the parent/child relation between artist and albums intact.

Picking Up Where Massive Left Off

Underlying the relational database integration in Biggy is re-worked version of another of Rob's projects, Massive. We've re-worked some of the internals, and added strong support for static types (the original Massive was all dynamic). Much of the speed and power of Biggy with respect to relational data comes from the innate flexibility of Massive, coupled with some customization to meet the needs of Biggy.

In a way, one might say that Biggy is the next layer that Massive was always waiting for. With the addition of static type handling and easy, synchronized in-memory persistence, Biggy helps take Massive to the next level.

Check it Out, but Be Warned

There's a lot going on with Biggy right now, as I write this. We are exploring a new architecture proposed by none other than K. Scott Allen ( @OdeToCode on Twitter), and things are in flux. That said, if you enjoy messing about with innovative persistence and data access technologies, do visit Github and check out the code, pull it down, and play with it. We need to know how it works "in the wild" and under conditions we haven't thought of.

But, be warned. As I said, things are moving fast, and as far as I know, the API is not locked in yet.

What's in it for Me?

This is the first Open Source Project I have really jumped in to. As I mentioned earlier, the scope was right, the technology I know well, and I think that my views on data access jibe well with the project owner - Rob does interesting things that challenge convention, and I totally subscribe to that. I have been looking for a project to jump on for a while, and this was perfect.

I have been coding for myself (and some odd projects related to my day job) for a few years, and until now have never really worked with someone else's code. I recognized early on that the thing to do is to try to ascertain and stay in tune with the project owner's direction. In fact, this is one of the more interesting aspects of participation, actually. I am accustomed to working my project out my own way, to suit my needs. I have never really had to try and follow someone's "vision" other than my own "how can I make this work" approach.

In the few short weeks I have been contributing to Biggy, I have already grown as a coder, and I intend to keep it up.

What's Next?

Over the next few posts, I'll be taking a look at some of the more interesting things I have run into on this project, and some of the things I have learned. Hope you check out Biggy, and follow along (we can ALWAYS use GOOD bug reports - at present, I am certain there are plenty to be had!).

Resources and Items of Interest

 

Posted on March 22 2014 04:58 PM by jatten     

Comments (2)

C#: Using Reflection and Custom Attributes to Map Object Properties

Posted on March 10 2014 05:40 PM by jatten in C#, CodeProject   ||   Comments (0)

map-to-the-cyan-studio-500I have a general distaste for decorating my code with Attributes and Annotations. Most of the time, I can't help but feel like there must be a better way to accomplish what I am trying to do, and/or that I have somewhere sprung a leak in what should be a helpful abstraction.

Other times, though, custom attributes can be just the tool for the job, and sometimes, the only practical way to solve a problem.

An easy to understand use case for Custom Attributes might be the mapping of object properties to database fields in a data access layer. You have no doubt seen this before when using Entity Framework. In EF, we often utilize System.ComponentModel.DataAnnotations to decorate the properties of our data objects.

Image by Elizabeth Briel | Some Rights Reserved

Here, we're going to take a quick look at creating our own custom attributes.

Use Custom Attributes to Give Hints or Property Metadata

Yes, EF and the System.ComponentModel.DataAnnotations namespace provide a ready-made means to do this, but for one, you may find yourself building your own data access layer or tool, and for another, this is an easy-to-understand example case.

Let's see how we might implement our own version of these data annotations as Custom Attributes. To create a Custom Attribute in C#, we simply create a class which inherits from System.Attribute. For example, if we wanted to implement our own [PrimaryKey] Attribute to indicate that a particular property on a class in our application represents the Primary Key in our database, we might create the following Custom Attribute:

public class PrimaryKeyAttribute : Attribute { }

 

Now, consider a class in our application, the Client class. Client has a ClientId property which corresponds to the Primary Key in the Clients database table:

public class Client 
{
    public int ClientId { get; set; }
    public string LastName { get; set; }
    public string FirstName { get; set; }
    public string Email { get; set; }
}

 

We could simply decorate the ClientId property with our new attribute, and then access this from code as in the simple example following:

Decorate the ClientId property with the Custom Primary Key Attribute:
public class Client
{
    [PrimaryKey]
    public int ClientId { get; set; }
    public string LastName { get; set; }
    public string FirstName { get; set; }
    public string Email { get; set; }
}

 

Use Reflection to Examine the Properties and Attributes of an Object

Then, we can cook up a simple console app demo to see how this works. First, we'll take the long way around, iterating using a foreach structure so we can see more clearly what's going on. Then we will look at a more concise (and efficient) LINQ-based implementation.

Silly Example of Accessing a Custom Property:
static void WritePK<T>(T item) where T : new()
{
    // Just grabbing this to get hold of the type name:
    var type = item.GetType();
  
    // Get the PropertyInfo object:
    var properties = type.GetProperties();
    Console.WriteLine("Finding PK for {0}", type.Name);
    foreach(var property in properties)
    {
        var attributes = property.GetCustomAttributes(false);
        foreach(var attribute in attributes)
        {
            if(attribute.GetType() == typeof(PrimaryKeyAttribute))
            {
            string msg = "The Primary Key for the {0} class is the {1} property";
            Console.WriteLine(msg, type.Name, property.Name);
            }
        }
    }
}

 

In the code above, we pass in a Generic object of type T (meaning, this method could be used with ANY domain object to check for the presence of a [PrimaryKey] attribute). We first use the GetType() method to find the object's Type information, and then we call the GetProperties() method of the Type instance, which returns an array of PropertyInfo objects.

Next, we iterate over each of the PropertyInfo instances, and call the GetCustomAttributes() method, which will return an array of objects representing the CustomAttributes found on that property. We can then check the type of each CustomAttribute object, and if it is of type PrimaryKeyAttribute, we know we have found a property that represents a primary key in our database.

LINQ Makes it Mo' Bettah

We could re-write the code above, using LINQ, for a more compact and efficient method as follows:

The WritePk Method, Re-Written Using LINQ:
static void WritePK<T>(T item) where T : new()
{
    var type = item.GetType();
    var properties = type.GetProperties();
    Console.WriteLine("Finding PK for {0}", type.Name);
    // This replaces all the iteration above:
    var property = properties
        .FirstOrDefault(p => p.GetCustomAttributes(false)
            .Any(a => a.GetType() == typeof(PrimaryKeyAttribute)));
    if (property != null)
    {
        string msg = "The Primary Key for the {0} class is the {1} property";
        Console.WriteLine(msg, type.Name, property.Name);
    }
}

 

This example is fairly simplistic, but illustrates well how we can access CustomAttributes to useful end. 

Another case, which I ran into recently is mapping properties to database columns. In creating a general-purpose data access tool, you never know how database columns are going to align with the properties on your domain objects. In my case, we needed to dynamically build some SQL, using reflection to grab object properties, and map to the database. However, there is no guarantee that the database column names will match the property names on the domain object.

In cases where column names differ from object properties in such a situation, Custom Attributes are one means of dealing with the situation (this is the part where the abstraction layer of the data access tool gets violated by the Db rearing its head into the business object domain . . .).

Use Custom Attributes to Map Properties to Database Columns

The previous example simply used a Custom Attribute simply as sort of a tag on a property. Attributes can also convey information if needed. Let's consider a means to map the property to a specific database column name.

Once again, we create a class which inherits from System.Attribute, but this time we will add a property and a constructor:

The Custom DbColumn Attribute:
public class DbColumnAttribute : Attribute
{
    string Name { get; private set; }
    public DbColumnAttribute(string name)
    {
        this.Name = name;
    }
}

 

Now, let's pretend you inherit a database which you need to integrate with your existing code base. The table from which the client information will be sourced uses all lower-case column names, with underscores between segments instead of proper or camel casing:

SQL For a Table With Column Names Which Do Not Match Class Properties:
CREATE TABLE Clients (
    client_id int IDENTITY(1,1) PRIMARY KEY NOT NULL,
    last_name varchar(50) NOT NULL,
    first_name varchar(50) NOT NULL,
    email varchar(50) NOT NULL
);

 

Now you can do this:

The Client Class with Column Name Attributes:
public class Client
{
    [PrimaryKey]
    [DbColumn("client_id")]
    public int ClientId { get; set; }
  
    [DbColumn("last_name")]
    public string LastName { get; set; }
  
    [DbColumn("first_name")]
    public string FirstName { get; set; }
  
    [DbColumn("email")]
    public string Email { get; set; }
}

 

You can access these attributes, and their properties, from code like so:

Reading Custom Attribute Properties from Code:
static void WriteColumnMappings<T>(T item) where T : new()
{
    // Just grabbing this to get hold of the type name:
    var type = item.GetType();
  
    // Get the PropertyInfo object:
    var properties = item.GetType().GetProperties();
    Console.WriteLine("Finding properties for {0} ...", type.Name);
    foreach(var property in properties)
    {
        var attributes = property.GetCustomAttributes(false);
        string msg = "the {0} property maps to the {1} database column";
        var columnMapping = attributes
            .FirstOrDefault(a => a.GetType() == typeof(DbColumnAttribute));
        if(columnMapping != null)
        {
            var mapsto = columnMapping as DbColumnAttribute;
            Console.WriteLine(msg, property.Name, mapsto.Name);
        }
    }
}

 

"But John," you say, "Entity Framework already does this!"

Precisely. But now you know how it works. Believe me, you may not always have EF at your disposal. Also, you WILL run into databases "in the wild" where column naming conventions do not align with C# Class and property naming conventions (Work with a Postgresql database for five minutes, and get back to me).

Cache Results from Calls Using Reflection Where Appropriate

A quick note, which is only marginally applicable to the examples above, but important in the design of a real-world application. Calls to using reflection can be expensive. Overall, machines these days are fast, and generally, the odd call to GetType() and GetCustomAttributes() are not all that significant. Except when they are.

For example, if the above code were used in a larger application context repeatedly, it might be better to walk through the object properties at object initialization (or even at application load) and map the object properties for each to its respective column name and stash them all in a Dictionary<string, string>. Then, anywhere in your code where the mapping is needed, you can access the primary key name for a specific object by use of the property as a key.

How you do this and where would depend heavily on what you are doing, and the larger structure of your application. For an example of what I am talking about, check out the Biggy project, where I recently had to do this very thing.

Custom Attributes Can Be an Architectural Trade-Off

I was working on an open-source project recently, and the project maintainer wisely pointed out that column-mapping attributes such as the above are "the database pushing right on up through the abstraction." Which is true. In the case of mapping database columns to object properties, we are attempting to solve but one aspect of the age-old impedance mismatch problem faced by all Object-Relational Mapping (ORM) frameworks. It ain't always elegant, but sometimes, it is the only way.

None of this is to say that Custom Attributes are only useful in the context of mapping database columns. There are any number of potential use-cases. While I personally dislike cluttering up my code with attributes and annotations, there will be times when it is the best way to solve a problem.

The next time you find yourself wishing you could know something extra about a particular property or method, Custom Attributes are one more tool in your chest.

Additional Resources and Items of Interest

 

Posted on March 10 2014 05:40 PM by jatten     

Comments (0)

ASP.NET MVC 5 Identity: Implementing Group-Based Permissions Management Part II

Posted on February 19 2014 09:02 PM by jatten in ASP.NET MVC, ASP.Net, C#, CodeProject   ||   Comments (5)

locked-awayThis is the second part of a two-part series in which we figure out how to implement a basic Group-based permissions management system using the ASP.NET MVC 5 Identity system. In this series, we are building upon previous concepts we used in extending the IdentityUser class and implementing Role-Based application security, and also in extending and customizing the IdentityRole class.

In this series, we hope to overcome some of the limitations of the simple "Users and Roles" security model, instead assigning Roles ("permissions") to Groups, and then assigning one or more groups to each user.

Image by Kool | Some Rights Reserved

In the first installment, we figured out how to model our core domain objects for the purpose of extending the ASP.NET Identity system into a basic Group-Based Permissions management mode. We decided that Groups will be assigned various combinations of permissions, and Users are assigned to one or more groups. What we are referring to here as "permissions" are actually the familiar "Role" provided by the identity system, upon which the MVC authorization system depends for user authentication and application access authorization.

<--- Review Part I: Extending the Model

 

Up to this point, we have extended our domain model by adding a Group class, implemented many-to-many relationships between Users and Groups, as well as between Groups and Roles ("Permissions"). We created

Building the Example Application - Controllers, Views, and ViewModels

Picking up where we left off, we now need to add the functional components of our example application. Obviously, we need some controllers and Views, but before we can build those, we are going to add some ViewModels which will be consumed by the various Controllers, and passed to the Views.

Adding Group View Models

We are going to need a few new ViewModels to complete our implementation of Groups. Also, we no longer need the SelectUserRolesViewModel. We will now be assigning Users to Groups, instead of directly to Roles, so we can delete the code for that. For the sake of simplicity, we will go ahead and add all of our new ViewModels to the AccountViewModels.cs file, and then I will explain what we are doing with each on as we go.

First, open the AccountViewModels.cs file, find the code for SelectUserRolesViewModel, and delete it.

Next, add the following new ViewModels to the end of the AccountViewModels.cs file:

Add New Required ViewModels:
// Wrapper for SelectGroupEditorViewModel to select user group membership:
public class SelectUserGroupsViewModel
{
    public string UserName { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<SelectGroupEditorViewModel> Groups { get; set; }
  
    public SelectUserGroupsViewModel()
    {
        this.Groups = new List<SelectGroupEditorViewModel>();
    }
  
    public SelectUserGroupsViewModel(ApplicationUser user)
        : this()
    {
        this.UserName = user.UserName;
        this.FirstName = user.FirstName;
        this.LastName = user.LastName;
  
        var Db = new ApplicationDbContext();
  
        // Add all available groups to the public list:
        var allGroups = Db.Groups;
        foreach (var role in allGroups)
        {
            // An EditorViewModel will be used by Editor Template:
            var rvm = new SelectGroupEditorViewModel(role);
            this.Groups.Add(rvm);
        }
  
        // Set the Selected property to true where user is already a member:
        foreach (var group in user.Groups)
        {
            var checkUserRole =
                this.Groups.Find(r => r.GroupName == group.Group.Name);
            checkUserRole.Selected = true;
        }
    }
}
  
  
// Used to display a single group with a checkbox, within a list structure:
public class SelectGroupEditorViewModel
{
    public SelectGroupEditorViewModel() { }
    public SelectGroupEditorViewModel(Group group)
    {
        this.GroupName = group.Name;
        this.GroupId = group.Id;
    }
  
    public bool Selected { get; set; }
  
    [Required]
    public int GroupId { get; set; }
    public string GroupName { get; set; }
}
  
  
public class SelectGroupRolesViewModel
{
    public SelectGroupRolesViewModel()
    {
        this.Roles = new List<SelectRoleEditorViewModel>();
    }
  
  
    // Enable initialization with an instance of ApplicationUser:
    public SelectGroupRolesViewModel(Group group)
        : this()
    {
        this.GroupId = group.Id;
        this.GroupName = group.Name;
  
        var Db = new ApplicationDbContext();
  
        // Add all available roles to the list of EditorViewModels:
        var allRoles = Db.Roles;
        foreach (var role in allRoles)
        {
            // An EditorViewModel will be used by Editor Template:
            var rvm = new SelectRoleEditorViewModel(role);
            this.Roles.Add(rvm);
        }
  
        // Set the Selected property to true for those roles for 
        // which the current user is a member:
        foreach (var groupRole in group.Roles)
        {
            var checkGroupRole =
                this.Roles.Find(r => r.RoleName == groupRole.Role.Name);
            checkGroupRole.Selected = true;
        }
    }
  
    public int GroupId { get; set; }
    public string GroupName { get; set; }
    public List<SelectRoleEditorViewModel> Roles { get; set; }
}
  
  
public class UserPermissionsViewModel
{
    public UserPermissionsViewModel()
    {
        this.Roles = new List<RoleViewModel>();
    }
  
  
    // Enable initialization with an instance of ApplicationUser:
    public UserPermissionsViewModel(ApplicationUser user)
        : this()
    {
        this.UserName = user.UserName;
        this.FirstName = user.FirstName;
        this.LastName = user.LastName;
        foreach (var role in user.Roles)
        {
            var appRole = (ApplicationRole)role.Role;
            var pvm = new RoleViewModel(appRole);
            this.Roles.Add(pvm);
        }
    }
  
    public string UserName { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<RoleViewModel> Roles { get; set; }
}

 

Adding a Groups Controller

Before we can do much more with our application, we need to add a Groups controller and associated Views. We already have a Roles Controller from our previous article, so let's add one for Groups now. We begin with a pretty standard CRUD-type controller as might be generated by Visual Studio:

The basic Groups Controller:
public class GroupsController : Controller
{
    private ApplicationDbContext db = new ApplicationDbContext();
  
    [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
    public ActionResult Index()
    {
        return View(db.Groups.ToList());
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
    public ActionResult Details(int? id)
    {
        if (id == null)
        {
            return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
        }
        Group group = db.Groups.Find(id);
        if (group == null)
        {
            return HttpNotFound();
        }
        return View(group);
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    public ActionResult Create()
    {
        return View();
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    [HttpPost]
    [ValidateAntiForgeryToken]
    public ActionResult Create([Bind(Include="Name")] Group group)
    {
        if (ModelState.IsValid)
        {
            db.Groups.Add(group);
            db.SaveChanges();
            return RedirectToAction("Index");
        }
        return View(group);
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    public ActionResult Edit(int? id)
    {
        if (id == null)
        {
            return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
        }
        Group group = db.Groups.Find(id);
        if (group == null)
        {
            return HttpNotFound();
        }
        return View(group);
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    [HttpPost]
    [ValidateAntiForgeryToken]
    public ActionResult Edit([Bind(Include="Name")] Group group)
    {
        if (ModelState.IsValid)
        {
            db.Entry(group).State = EntityState.Modified;
            db.SaveChanges();
            return RedirectToAction("Index");
        }
        return View(group);
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    public ActionResult Delete(int? id)
    {
        if (id == null)
        {
            return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
        }
        Group group = db.Groups.Find(id);
        if (group == null)
        {
            return HttpNotFound();
        }
        return View(group);
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    [HttpPost, ActionName("Delete")]
    [ValidateAntiForgeryToken]
    public ActionResult DeleteConfirmed(int id)
    {
        Group group = db.Groups.Find(id);
        var idManager = new IdentityManager();
        idManager.DeleteGroup(id);
        return RedirectToAction("Index");
    }
  
  
    [Authorize(Roles = "Admin, CanEditGroup")]
    public ActionResult GroupRoles(int id)
    {
        var group = db.Groups.Find(id);
        var model = new SelectGroupRolesViewModel(group);
        return View(model);
    }
  
  
    [HttpPost]
    [Authorize(Roles = "Admin, CanEditGroup")]
    [ValidateAntiForgeryToken]
    public ActionResult GroupRoles(SelectGroupRolesViewModel model)
    {
        if (ModelState.IsValid)
        {
            var idManager = new IdentityManager();
            var Db = new ApplicationDbContext();
            var group = Db.Groups.Find(model.GroupId);
            idManager.ClearGroupRoles(model.GroupId);
            // Add each selected role to this group:
            foreach (var role in model.Roles)
            {
                if (role.Selected)
                {
                    idManager.AddRoleToGroup(group.Id, role.RoleName);
                }
            }
            return RedirectToAction("index");
        }
        return View();
    }
  
  
    protected override void Dispose(bool disposing)
    {
        if (disposing)
        {
            db.Dispose();
        }
        base.Dispose(disposing);
    }
}

 

By now, most of the code above should look fairly familiar. We have the basic Create/Edit/Delete and Index methods, and then one odd method at the end of the class, GroupRoles. Actually, this should look sort of familiar as well. This is a simple adaptation of the code we used in the previous project to select Roles for individual Users. Here, we are doing the same thing for specific Groups instead.

Add Views for the Groups Controller

The following are the Views we need for the Groups Controller. These are also pretty standard fare, except for the GroupRoles View. We'll include them all here for completeness, though.

Of greatest interest is the GroupRoles View, so we will start there.

The GroupRoles View

This View is where we will assign one or more Roles to a specific Group. We want to display the general information for the current group selected, and then display a list of all the available Roles, with checkboxes to indicate selected status. For our presentation layer, we will describe Roles as "Permissions" so that the concept is more clear to the user: Users are members of groups, and groups have sets of permissions.

Here, we once again employ an EditorTemplate (our SelectRoleEditorTemplate from the previous version of this project) in order to render an HTML Table with checkboxes to indicate selection status for each row item. This View is nearly identical to the View we used in the previous version of this project for for the UserRoles View (in fact, I simply made a few quick changes to that one, and renamed it)..

The GroupRoles View:
@model AspNetGroupBasedPermissions.Models.SelectGroupRolesViewModel
@{ ViewBag.Title = "Group Role Permissions"; }
  
<h2>Permissions for Group @Html.DisplayFor(model => model.GroupName)</h2>
<hr />
  
@using (Html.BeginForm("GroupRoles", "Groups", FormMethod.Post, new { encType = "multipart/form-data", name = "myform" }))
{
    @Html.AntiForgeryToken()
    <div class="form-horizontal">
        @Html.ValidationSummary(true)
        <div class="form-group">
            <div class="col-md-10">
                @Html.HiddenFor(model => model.GroupName)
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-10">
                @Html.HiddenFor(model => model.GroupId)
            </div>
        </div>
        <h4>Select Role Permissions for @Html.DisplayFor(model => model.GroupName)</h4>
        <br />
        <hr />
        <table>
            <tr>
                <th>
                    Select
                </th>
                <th>
                    Permissions
                </th>
            </tr>
            @Html.EditorFor(model => model.Roles)
        </table>
        <br />
        <hr />
        <div class="form-group">
            <div class="col-md-offset-2 col-md-10">
                <input type="submit" value="Save" class="btn btn-default" />
            </div>
        </div>
    </div>
}
<div>
    @Html.ActionLink("Back to List", "Index")
</div>

 

In the above, the line @html.EditorFor(model => model.Roles) causes the MVC framework to dig out our (Carefully named!!) SelectRoleEditorViewModel from the Views/Shared/EditorTempates/ directory, and uses that to render each Role item as a table row.

If you have been following this "series" of articles, this should be familiar territory by now.

From here, the rest of these views are rather standard fare.

The Index Group View

Code for the Index Group View:
@model IEnumerable<AspNetGroupBasedPermissions.Models.Group>
@{ ViewBag.Title = "Index"; }
  
<h2>Groups</h2>
  
<p>
    @Html.ActionLink("Create New", "Create")
</p>
<table class="table">
    <tr>
        <th>
            @Html.DisplayNameFor(model => model.Name)
        </th>
        <th></th>
    </tr>
@foreach (var item in Model) 
{
    <tr>
        <td>
            @Html.DisplayFor(modelItem => item.Name)
        </td>
        <td>
            @Html.ActionLink("Edit", "Edit", new { id=item.Id }) |
            @Html.ActionLink("Permissions", "GroupRoles", new { id=item.Id }) |
            @Html.ActionLink("Delete", "Delete", new { id=item.Id })
        </td>
    </tr>
}
</table>

 

Note above, we have added three ActionLinks at the end of each row - "Edit", "Permissions", and "Delete."

These will link us to the appropriate methods on the RolesController. Of specific interest is the "Permissions" link, which will direct us to our GroupRoles method, and allow us to assign one or more Roles ("Permissions") to each group. This, so to speak, is the business end of our authorization management.

The Create Group View

The Create Group View:
@model AspNetGroupBasedPermissions.Models.Group
@{ ViewBag.Title = "Create Groups"; }
  
<h2>Create a new Group</h2>
  
@using (Html.BeginForm()) 
{
    @Html.AntiForgeryToken()
    <div class="form-horizontal">
        <h4>Group</h4>
        <hr />
        @Html.ValidationSummary(true)
        <div class="form-group">
            @Html.LabelFor(model => model.Name, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                @Html.EditorFor(model => model.Name)
                @Html.ValidationMessageFor(model => model.Name)
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-offset-2 col-md-10">
                <input type="submit" value="Create" class="btn btn-default" />
            </div>
        </div>
    </div>
}
<div>
    @Html.ActionLink("Back to List", "Index")
</div>
@section Scripts { @Scripts.Render("~/bundles/jqueryval") }

 

The Edit Group View

Code for the Edit Group View:
@model AspNetGroupBasedPermissions.Models.Group
@{ ViewBag.Title = "Edit"; }
  
<h2>Edit</h2>
  
@using (Html.BeginForm())
{
    @Html.AntiForgeryToken()
    <div class="form-horizontal">
        <h4>Group</h4>
        <hr />
        @Html.ValidationSummary(true)
        @Html.HiddenFor(model => model.Id)
        <div class="form-group">
            @Html.LabelFor(model => model.Name, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                @Html.EditorFor(model => model.Name)
                @Html.ValidationMessageFor(model => model.Name)
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-offset-2 col-md-10">
                <input type="submit" value="Save" class="btn btn-default" />
            </div>
        </div>
    </div>
}
<div>
    @Html.ActionLink("Back to List", "Index")
</div>
@section Scripts { @Scripts.Render("~/bundles/jqueryval") }

 

The Delete Group View

Code for the Delete Group View:
@model AspNetGroupBasedPermissions.Models.Group
@{ ViewBag.Title = "Delete"; }
  
<h2>Delete</h2>
  
<h3>Are you sure you want to delete this?</h3>
<div>
    <h4>Group</h4>
    <hr />
    <dl class="dl-horizontal">
        <dt>
            @Html.DisplayNameFor(model => model.Name)
        </dt>
        <dd>
            @Html.DisplayFor(model => model.Name)
        </dd>
    </dl>
    @using (Html.BeginForm()) {
        @Html.AntiForgeryToken()
        <div class="form-actions no-color">
            <input type="submit" value="Delete" class="btn btn-default" /> |
            @Html.ActionLink("Back to List", "Index")
        </div>
    }
</div>

 

Update AccountController to Assign Users to Groups

Now that we have our Groups controller and Views in place, we need to update the code on our original AccountController. Previously, we have created a UserRoles method on AccountController, by which we assigned one or more Roles to a specific user. Now, instead, we are going to be assigning one or more Groups to specific User.

Open the AccountController file, and delete the UserRoles method, replacing it with the following code for UserGroups:

The UserGroups Method for AccountController:
[Authorize(Roles = "Admin, CanEditUser")]
public ActionResult UserGroups(string id)
{
    var user = _db.Users.First(u => u.UserName == id);
    var model = new SelectUserGroupsViewModel(user);
    return View(model);
}
  
  
[HttpPost]
[Authorize(Roles = "Admin, CanEditUser")]
[ValidateAntiForgeryToken]
public ActionResult UserGroups(SelectUserGroupsViewModel model)
{
    if (ModelState.IsValid)
    {
        var idManager = new IdentityManager();
        var user = _db.Users.First(u => u.UserName == model.UserName);
        idManager.ClearUserGroups(user.Id);
        foreach (var group in model.Groups)
        {
            if (group.Selected)
            {
                idManager.AddUserToGroup(user.Id, group.GroupId);
            }
        }
        return RedirectToAction("index");
    }
    return View();
}

Notice in the above, when we the HTTP Post is returned from the View, we need to clear all the User Group assignments, and then individually add the user to each of the groups selected in the ViewModel.

Replace the UserRoles View with a UserGroups View

With our new UserGroups method in place on AccountController, we need to replace the former UserRoles View with a very similar UserGroups View. As previously, we will display the basic User data for a specific, user, along with a list of available Groups to which the the User might be assigned. Once again, using a table with checkboxes, we can assign the user to one or more Groups.

As with the now-deprecated UserRoles View, and also the newly added GroupRoles View, we need a special Editor Template ViewModel and a correspondingly-named Editor Template View to represent each Group in the table. We have already added a SelectGroupEditorViewModel to the AccountViewModels.cs file. Now we need to add the corresponding Editor Template View.

Add the following View to the Views/Shared/EditorTemplates/ directory.Be careful to name it SelectGroupEditorViewModel so that it exactly matches the name of the ViewModel it represents:

The SelectGroupEditorViewModel View:
@model AspNetGroupBasedPermissions.Models.SelectGroupEditorViewModel
@Html.HiddenFor(model => model.GroupId)
<tr>
    <td style="text-align:center">
        @Html.CheckBoxFor(model => model.Selected)
    </td>
    <td style="padding-right:20px">
        @Html.DisplayFor(model => model.GroupName)
    </td>
</tr>

 

Now, with the in place, delete the old UserRoles View from the Views/Account/ directory, and add a new View named UserGroups as follows:

The UserGoups View:
@model AspNetGroupBasedPermissions.Models.SelectUserGroupsViewModel
@{ ViewBag.Title = "User Groups"; }
  
<h2>Groups for user @Html.DisplayFor(model => model.UserName)</h2>
<hr />
  
@using (Html.BeginForm("UserGroups", "Account", FormMethod.Post, new { encType = "multipart/form-data", name = "myform" }))
{
    @Html.AntiForgeryToken()
    <div class="form-horizontal">
        @Html.ValidationSummary(true)
        <div class="form-group">
            <div class="col-md-10">
                @Html.HiddenFor(model => model.UserName)
            </div>
        </div>
        <h4>Select Group Assignments</h4>
        <br />
        <hr />
        <table>
            <tr>
                <th>
                    Select
                </th>
                <th>
                    Group
                </th>
            </tr>
            @Html.EditorFor(model => model.Groups)
        </table>
        <br />
        <hr />
        <div class="form-group">
            <div class="col-md-offset-2 col-md-10">
                <input type="submit" value="Save" class="btn btn-default" />
            </div>
        </div>
    </div>
}
<div>
    @Html.ActionLink("Back to List", "Index")
</div>

 

Update the Action Links on the Account/Index View

We need to make a minor update to the Account Index View. Currently, the links next to each User in the table indicate "Roles" and point to the (now non-existent) UserRoles method. Instead, we will display the text "Groups" and point the link at the newly added UserGroups method. The modified code should look like the following:

Update the ActionLinks for the Table Items in the Account/Index View:
@model IEnumerable<AspNetGroupBasedPermissions.Models.EditUserViewModel>
@{ ViewBag.Title = "Index"; }
<h2>Index</h2>
<p>
    @Html.ActionLink("Create New", "Register") 
</p>
<table class="table">
    <tr>
        <th>
            @Html.DisplayNameFor(model => model.UserName)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.FirstName)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.LastName)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.Email)
        </th>
        <th></th>
    </tr>
@foreach (var item in Model) {
    <tr>
        <td>
            @Html.DisplayFor(modelItem => item.UserName)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.FirstName)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.LastName)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.Email)
        </td>
        <td>
            @Html.ActionLink("Edit", "Edit", new { id = item.UserName }) |
            @Html.ActionLink("Groups", "UserGroups", new { id = item.UserName }) |
            @Html.ActionLink("Delete", "Delete", new { id = item.UserName })
        </td>
    </tr>
}
</table>

 

Update Navigation Links on _Layout.cshtml

Now we just need to make sure we can access all of the new functionality we just built into our application. In the middle of the code on the _Layout.cshtml file, we need to update the Navigation links to match this:

Updated Navigation Links on _Layout.cshtml
<div class="navbar-collapse collapse">
    <ul class="nav navbar-nav">
        <li>@Html.ActionLink("Home", "Index", "Home")</li>
        <li>@Html.ActionLink("About", "About", "Home")</li>
        <li>@Html.ActionLink("Contact", "Contact", "Home")</li>
        <li>@Html.ActionLink("Users", "Index", "Account")</li>
        <li>@Html.ActionLink("Groups", "Index", "Groups")</li>
        <li>@Html.ActionLink("Permissions", "Index", "Roles")</li>
    </ul>
    @Html.Partial("_LoginPartial")
</div>

 

Setting up Initial Authorization Permissions

What is not obvious in the code here (unless you cloned the completed project) is that we are implementing our new Users/Groups/Permissions model upon the controllers we just created. Given the example Roles ("Permissions" I included in the Migrations Configuration file (what we are "Seeding" the database with), I set up the initial method-level [Authorize] attributes using the following permissions scheme. If it is not already, add the following [Authorize] Attributes to the appropriate method on each controller. Replace any that exist from the previous project.

Account Controller [Authorize] Roles:

Action Method

Roles Allowed
Login [AllowAnonymous]
Register [Authorize(Roles = "Admin, CanEditUser")]
Manage [Authorize(Roles = "Admin, CanEditUser, User")]
Index [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
Edit [Authorize(Roles = "Admin, CanEditUser")]
Delete [Authorize(Roles = "Admin, CanEditUser")]
UserGroups [Authorize(Roles = "Admin, CanEditUser")]

 

Groups Controller [Authorize] Roles:

Action Method

Roles Allowed
Index [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
Details [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
Create [Authorize(Roles = "Admin, CanEditGroup")]
Edit [Authorize(Roles = "Admin, CanEditGroup")]
Delete [Authorize(Roles = "Admin, CanEditGroup")]
GroupRoles [Authorize(Roles = "Admin, CanEditGroup")]

 

Roles Controller [Authorize] Roles:

Action Method

Roles Allowed
Index [Authorize(Roles = "Admin, CanEditGroup, CanEditUser")]
Create [Authorize(Roles = "Admin")]
Edit [Authorize(Roles = "Admin")]
Delete [Authorize(Roles = "Admin")]

 

As we can see, I have done my best (within space and the practical constraints of this already lengthy example project) to structure a tiered authorization scheme, modestly following a sort of "Principle of Least Privilege." In a production application, we can assume you would have additional business domains, and would need to think through the Role permission assignments with care.

Running the Application

If we start our application, we should be greeted with the standard Login screen. Once logged in, if we navigate to the Groups Link, we should be greeted with a list of the Groups we seeded our database with:

The Groups Screen:

application-groups_thumb3

If we click on the "Permissions" link, we find the roles, or "permissions" currently assigned to a particular group:

Permissions Screen:

application-group-roles_thumb3

If we navigate to the Users screen, we see what we might expect - a list of users, with the option to drill down and see which Groups a specific User belongs to:

The Users Screen:

application-users_thumb2

What might be Handy, though, is an additional link for each listed user whereby we can see what permissions they have as a result of all the groups in which they participate.

Adding a View For User Effective Permissions

Fortunately, we can do just that. First, we need to add one more link to the Accounts/Index View. Near the bottom, where we set up the links next to each row of table data, add the link as below for "Effective Permissions:

Add Effective Permissions Link to Account/Index View:
<td>
    @Html.ActionLink("Edit", "Edit", new { id = item.UserName }) |
    @Html.ActionLink("Groups", "UserGroups", new { id = item.UserName }) |
    @Html.ActionLink("Effective Permissions", "UserPermissions", new { id = item.UserName }) ||| 
    @Html.ActionLink("Delete", "Delete", new { id = item.UserName })
</td>

 

We have already added the UserPermissionsViewModel to AccountViewModels.cs, so now we just need to add a UserPermissions view to the Views/Account directory:

The UserPermissions View:
@model AspNetGroupBasedPermissions.Models.UserPermissionsViewModel
@{ ViewBag.Title = "UserPermissions"; }
  
<h2>Effective Role Permissions for user: @Html.DisplayFor(model => model.UserName)</h2>
<hr />
  
<table class="table">
    <tr>
        <th>
            Role Permission
        </th>
        <th>
           Description
        </th>
        <th></th>
    </tr>
    @foreach (var item in Model.Roles)
    {
        <tr>
            <td>
                @Html.DisplayFor(modelItem => item.RoleName)
            </td>
            <td>
                @Html.DisplayFor(modelItem => item.Description)
            </td>
        </tr>
    }
</table>
<div>
    @Html.ActionLink("Back to List", "Index")
</div>

 

Now, if we run our application, we can drill down and see all of the permissions afforded a given user as a result of all the groups in which that user participates:

Navigate to Users to Find Effective Permissions Link:

users-before-effective-permission-se[1]

 

View Effective Permissions for the Selected User across All Ggroups:

users-after-effective-permission-sel[2]

Some Thoughts About Authorization Management and Your Website

The ASP.NET Identity system affords us an easy to use abstraction over a complicated topic, and one which is at the forefront of today's news ("Ripped from today's headlines" so to speak). The ASP.NET Identity system represents one option for securing your site and managing authentication, but it is not the only way. The ASP.NET team presents several options for site security, and in fact Visual Studio gives you several choices as part of setting up your MVC project.

The Identity system appears to be a good choice for public-facing websites, integration with social media providers, and sites with simpler permissions management needs. Other options for managing site security include Active directory integration, and Windows Authentication for intranet-based services.

Of these options, the Identity system is the easiest to implement, and is built-in to the project templates included with ASP.NET and MVC.

As mentioned repeatedly throughout this article, the more finely-grained your security system can be, the more control you have. However, that control comes at the price of complexity. Not just in terms of the code, but also in terms of managing the various Users, Groups, and permission sets you create.

I guarantee you don't want to be sprinkling new roles willy-nilly throughout your application code, and then trying to manage them later. Plot it out ahead of time, with your business domains firmly in mind. Strike a balance between manageability and the Principle of Least Privilege.

The example solution presented here offers a starting point. Keep in mind, though, that the hard-coded nature of the security protections using the [Authorize] attribute could easily become a code-maintenance nightmare if you get carried away. Also, adding the ability to create/edit "permissions" (which. remember, are actually "Roles" so far as the Identity framework is concerned) introduces a new convenience, and also a new dimension for trouble.

I recommend that the creation, editing, and deletion of roles be limited to application developers only (create a special role/group just for them!), and only really as a means to add roles to the database after first adding them to the appropriate [Authorize] attributes in the code.

I am most interested to hear feedback on this. Whether this was the exact solution you have been looking for, or you have spotted an idiotic, gaping flaw in my reasoning. Please feel free to comment, or shoot me an email at the address under the "About the Author" sidebar.

Additional Resources and Items of Interest

 

Posted on February 19 2014 09:02 PM by jatten     

Comments (5)

About the author

My name is John Atten, and my "handle" on many of my online accounts is xivSolutions. I am Fascinated by all things technology and software development. I work mostly with C#, JavaScript/Node, and databases of many flavors. Actively learning always. I dig web development. I am always looking for new information, and value your feedback (especially where I got something wrong!). You can email me at:

jatten at typecastexception dot com

Web Hosting by