LJ Archive

RedHen CRM—an Open-Source CRM Solution Built Entirely with Drupal

Sean Larkin

Lev Tsypin

Issue #888, April 2068

Historically difficult to implement in Drupal, native CRM solutions in Drupal 7 now allow site builders to craft more personalized user experiences for their Web site visitors by integrating their Drupal CMS with their customer relationship management system (CRM).

One of Drupal's key value propositions is its ability to integrate conventional content management features with other Web-based applications and tools. More and more, Drupal is becoming a Web application platform for building engaging Web sites that provide rich, personalized content experiences for site visitors.

We often recommend that our clients begin crafting more personalized user experiences for their Web site visitors by integrating their Drupal CMS with their customer relationship management system (CRM).

The benefits of such integrations are many. You can present forms for your site visitors to update their own CRM contact records or to provide you with better data about their interests and communication preferences. Based upon such information, you can serve up more pertinent content, as well as offer premium access to special features.

In fact, the value of CRM/CMS integration is becoming so compelling that most large CRM companies, such as Salesforce, Microsoft Dynamics, Oracle and Blackbaud, are enhancing their Web-based products and adjusting their marketing language to describe their offerings as “social enterprise” or “customer engagement” platforms. The pundits often use the buzzwords “Social CRM” to describe these retooled and rebranded CRM solutions.

The State of Third-Party CRM Integration in Drupal

Drupal has long been ahead of the CRM integration curve because of its open-source flexibility. Integration between Drupal and the open-source CRM CiviCRM (civicrm.org) historically has seen the widest adoption. Like Drupal, CiviCRM runs on a LAMP stack (Linux, Apache, MySQL and PHP) and can be configured to run as a component of a Drupal Web site. CiviCRM is a mature solution with a number of great features mostly geared toward the constituent relationship management needs of nonprofit organizations.

While CiviCRM's integration with Drupal is robust, it's still not perfectly seamless due to the fact that CiviCRM was engineered to work with multiple CMS front ends. CiviCRM relies on its own database schema and APIs. It also leverages its own templating engine that presents different markup from that generated by Drupal. For Drupal developers looking to build tightly integrated CMS/CRM solutions, CiviCRM integration requires mastering more APIs and maintaining more code. For site builders and Web site administrators, CiviCRM integration requires additional training, because CiviCRM's administrative interfaces follow different design patterns from those provided by Drupal. Further, given that major releases of CiviCRM rely upon specific versions of Drupal core, upgrading Drupal-CiviCRM solutions can present significant hurdles.

CRM Options Built Natively within Drupal

The Drupal community has long aspired for native CRM functionality. The most obvious benefits of native CRM in Drupal include:

  • A more seamless user experience for site visitors registering for events, making donations/payments or engaging in other Web site transactions.

  • The opportunity to leverage Drupal's growing suite of mobile and responsive tools and themes for CRM interfaces.

  • The ability to expose CRM data as content and display aggregate CRM data on your site.

  • Increased opportunities to integrate CRM data with Drupal contributed modules, such as data visualization and geo-mapping.

  • Reduced staff training costs by eliminating the need for staff training on multiple platforms.

  • Potential reductions in technical risk since all your tools rely on a single Drupal instance.

  • Potential reductions in hosting and IT costs.

  • The ability to do complex engagement scoring or engagement analytics (more on this later).

  • And probably most important, the ability to customize fully your CRM solution “the Drupal way”.

The arrival of the entity framework in Drupal 7 core has opened the door to the development of more robust CRM solutions built natively in Drupal. So, after a year of scheming and more than 1,000 hours of intense development, ThinkShout, Inc., recently announced the beta release of RedHen CRM—a native CRM solution written for Drupal in Drupal. And, we anticipate that RedHen will have a stable release by the time this article is published.

RedHen CRM as a CRM “Framework”

ThinkShout initially designed RedHen CRM around the complex association management (AMS) needs of nonprofits and trade associations. Based upon a site visitor's affiliation with an organization that has purchased one or more membership subscriptions, that site visitor is provided premium access to content, e-commerce discounts and advanced Web site features when logged in to a Web site built on top of RedHen CRM.

These association management requirements go well beyond the standard needs of most CRM/CMS solutions. However, in building an abstract tool with these needs at the forefront, our goal has been to future-proof RedHen CRM as much as possible to ensure that its architecture can accommodate a wide variety of use cases.

RedHen CRM is similar to Drupal Commerce (drupal.org/project/commerce) in its modular structure. As with Drupal Commerce, the core RedHen modules that can be downloaded on the drupal.org project page won't provide you with a working CRM right out of the box. RedHen is intended to provide developers and site builders with the building blocks for quickly creating their own CRM data models that map to their particular business requirements and workflow. Consequently, setting up a new RedHen CRM instance does require configuration.

In the future, ThinkShout is likely to release RedHen “Features” or “Apps” that provide prepackaged CRM solutions for different use cases. At the time of this writing, we know of more than 96 RedHen CRM solutions built by other Drupal developers and site builders, and we anticipate this number will increase by the time this article is published.

In addition to its association management uses, we see RedHen CRM as an ideal starting point for building custom sales pipeline management tools and project management applications, as well as Drupal integration points with third-party ERP (enterprise resource planning) tools and financial accounting packages.

RedHen CRM Project Structure

RedHen CRM consists of a core module containing shared APIs and interfaces and a collection of feature-specific sub-modules, including:

  • Contact (redhen_contact): contact entities and APIs, along with integration with Drupal users.

  • Fields (redhen_fields): custom field types used by RedHen entities. Currently this includes a unique e-mail field, which assigns attributes to e-mails, such primary, bulk and custom labels (home, work and so on).

  • Organization (redhen_org): organization entities and APIs.

  • Organization Group (redhen_group): lightweight group feature that turns organizations into containers for private content with a broadcast system.

  • Relations (redhen_relation): the Relation module allows connections between contacts and organizations.

  • Note (redhen_note): notes on contacts and organizations.

  • Engagement (redhen_engagement): engagement scoring system and APIs. RedHen Note integration is included, as well as Rules integration for popular modules, such as Comment, Registration and Webform.

  • Registration (redhen_registration): integration with the Entity Registration module.

We maintain other functionality that is not needed for all uses cases in separate projects in order to keep the core RedHen code base as lean as possible. The RedHen Membership system (drupal.org/project/redhen_membership) is our most widely used RedHen component with its own project name space. It handles individual and organizational membership subscriptions. As with Drupal Commerce, key sub-modules will continue to be included with the main module code base. However, we anticipate that as the RedHen CRM developer community grows, we will see more and more contributed modules that extend RedHen's core feature set.

The overall architecture of RedHen CRM consists of a set of minimalistic building blocks that developers and site builders can use to develop solutions tailored to specific use cases. Like Drupal Commerce, RedHen relies on Drupal distributions and installation profiles for the heavy lifting of fleshing out polished applications that serve specific needs.

RedHen CRM Module Dependencies

RedHen has minimal dependencies on other Drupal contributed modules, striving for a balance between relying on its own code base and leveraging other contributed modules. Beyond Drupal core, RedHen CRM currently depends only on Entity API, discussed in detail below and the Relation module (drupal.org/project/relation) for managing connections. RedHen CRM supports integration with popular contributed tools, like Views and Rules, but these modules are not required. This makes RedHen a leaner and more stable application platform, as it relies less on a shifting foundation of other contributed modules over which we have little to no control.

This centrist approach helps assure other Drupal developers that their customizations and extensions of RedHen won't break due to fragile module interdependencies. At the same time, it provides less-technical site builders with a known approach to extending RedHen CRM with standard Drupal tools, such as Views, Rules and the Fields API.

Custom Entities

RedHen CRM relies on custom Drupal entity types and bundles. It leans heavily on Entity API (drupal.org/project/entity), a wrapper around Drupal's core entity system that eases developing entities in several ways. The Entity API module:

  • Provides classes and controllers to streamline entity creation and management. RedHen extends these base classes as needed.

  • Eases integration with key contributed modules, namely Views (drupal.org/project/views) and Rules (drupal.org/project/rules).

  • Exposes standardized API hooks during CRUD operations providing a consistent interface for other modules to interface with your custom entities.

  • Provides hooks and data structures allowing for entities to be exported, for example, using Features or CTools. RedHen leverages this feature to make entity bundle definitions, such as types of contacts and organizations, exportable.

RedHen ships with the following entity types, each of which comes bundled with core properties and can be extended with additional user-defined fields:

  • Contacts

  • Organizations

  • Notes

  • Memberships (part of RedHen Membership)

  • Engagements

Connections

A key feature of any CRM is managing connections between organizations and contacts. RedHen features a very flexible connection system built on top of the Relation module, which allows for bidirectional connections between arbitrary types of entities. These relationships themselves can have additional user-defined fields. RedHen ships with two types of relationships: Affiliations, which are connections between an organization and a contact, and Personal Connections, which are connections between contacts. These relationships come bundled with status and role properties, but fields can be added that are applicable to a given relationship.

For example, imagine that you want to define a relationship between a contact of the type “Staff” and an organization of the type “Company”. Suppose you want to include information about the position that contact has at that organization. This field value has meaning only within the context of relationship, and therefore, this data is stored with the relationship rather than on the contact record or the organization record itself.

Figure 1. Listing of a Contact's Connections

Building an Example Application with RedHen

Now that you've learned about RedHen, you most likely will want to know the steps involved in building a useful application with it. Below is a step-by-step tutorial for setting up an association management system (AMS) for the National Association of Pet Shelters, an fictional nonprofit organization. For those who want to follow along, the complete demo application is available for download as an installation profile called RedHen Demo (drupal.org/project/redhen_demo) on drupal.org.

Installation

Here are the installation steps:

  1. Download the latest version of Drupal 7, RedHen, RedHen Membership, Entity API, Relation and Views.

  2. Install Drupal using an installation profile of your choice.

  3. Enable RedHen and its dependencies.

Structure

When RedHen is enabled by itself, it defines types of entities, but it does not create instances of these entity types. That is because, out of the box, RedHen doesn't presume to know the types of contacts, organization, memberships and so on, that will be needed for a given CRM deployment. In Drupal speak, these entity type instances are called bundles. In the next step, you will define bundles for each of your RedHen entity types.

First, navigate to /admin/structure/redhen to see the types of entities that can be managed through RedHen. They are, again, contacts, engagement scores, memberships, notes and organizations. For each of them, create the following bundles, respectively:

  • Contacts: contacts have only a label and machine name. Create one for Staff and Volunteers.

  • Organizations: aside from machine name and label, organization bundles can be turned into content groups if the redhen_org_group module is enabled. Checking the “groupify” box exposes two additional settings: private and a list of content types that can be posted into the group. Create two organizations, Foundation and Shelter, making the latter a content group.

  • Notes: notes come with a default single bundle since there is only one type of note.

  • Memberships: memberships have an additional user role property that is inherited by linked Drupal users with a given membership. Create Premium and Standard memberships.

  • Engagement Scores: engagement scores have an additional “score” property. Create High-value engagement and Standard-value engagement with arbitrary scores.

Now that your bundles have been created, you can add fields to them. Each bundle has a “manage fields” link. To manage fields for your staff contact type, for example, visit /admin/structure/redhen/contact_types/manage/staff/fields. You'll notice the e-mail field, which RedHen adds automatically behind the scenes to each contact bundle, because e-mail is required for several core components of the system. Incidentally, we developed our own e-mail field that mimics v-card structure allowing for multiple e-mail addresses per contact that have different labels and flags for default, bulk and on-hold statuses. In addition, the contact name is visible, although it doesn't have any settings to manage. This is because name is a property on a contact, which Drupal allows developers to expose as a “pseudo field”, so that its position can be managed along with other traditional fields.

Figure 2. Managing the Fields Associated with a Contact Record

The interface for managing fields is part of Drupal core and is available when the Field UI module is enabled. Any field type can be added to your contact bundle, ranging from a simple text field to a complex address field, to an image or file upload field. Once a field is added to a bundle, it will be presented automatically when adding or editing an instance of the entity bundle (that is, an individual contact record), and it will be available to other components, such as Views or Rules. Fields attached to RedHen entities do not get added to the default tabular listings, but they are made available automatically as filters when changing bundles. You can override the default listings by creating a custom View with the same path and including any additional fields.

Figure 3. Custom Fields Being Exposed as Filters on a Listing of Contacts

Relationships are another key component of RedHen and can connect contacts to organizations and other contacts. As mentioned earlier, RedHen installs two types of relations: Affiliations for connecting contacts to organizations and Personal Connections for relating contacts to each other. From the Relation types interface, you can add fields to these default relation types—for example, a position field to Affiliation—or create additional relation types, such as Employee.

When creating a relation, you must specify the types of source and target entities and directionality of the relation. For the sake of this demo, let's stick with the default relation types.

Figure 4. Listing of Default RedHen Relation Types

Playing Well in the Drupal Sandbox

RedHen's default listings of contacts are a good start, but the fun really starts if you want to create your own custom views. The venerable Views module, a powerful graphical query builder, is the gold standard in Drupal for creating custom “lists of stuff”, including RedHen entities. Much of the heavy lifting involved in Views integration is handled by the Entity API, but any module implementing the Entity API interfaces still needs to clarify the data types of all entity properties and provide Views handlers for any nonstandard data. For example, organization entities have a primary contact associated with them, and it's up to RedHen to explain to Views that the related contact ID within an organization record is actually a RedHen contact. Let's create a new View, adding additional fields and relationships to the standard list of contacts:

  1. Create a new view at /admin/structure/views/add of Contacts.

  2. Add relationships to Memberships and Organization affiliation.

  3. Add the following fields: contact first and last name, contact e-mail (change the format to primary e-mail), membership name and organization name.

You now have a View of all contacts that includes their membership and organization. If you set the path of the View to /redhen/contact, it simply will override the default contacts listing, or you can move it elsewhere to maintain both interfaces.

Figure 5. Custom Contact View with an Address and Exposed Filters

Similarly, business rules can be extended using the Rules module, which provides an interface for defining logic based upon a trigger→action model. RedHen again leans on the Entity API to expose RedHen entities into this model, so that you can, for example, send an e-mail to a contact when a related membership entity is updated.

Figure 6. Custom Rule That Demonstrates Sending an E-mail to New Contacts

Extending Core Functionality

RedHen also can be extended and customized the old-fashioned way—by writing custom code. All of RedHen's entity types feature APIs with standard CRUD operations and wrappers around many common tasks, such as getting all related entities. In addition, RedHen can be manipulated using Drupal's hook system. Each entity can be altered using a hook implementation when it's loaded, created, updated and deleted. Most hooks are exposed through the Entity API's inherited controller classes, but RedHen features some of its own unique hooks as explained in redhen.api.php. For example, a module can dictate whether a contact entity can be deleted by implementing hook_redhen_contact_can_delete():

function mymodule_redhen_contact_can_delete(RedhenContact $contact) {
  // prevent the deletion of active contacts
  if ($contact->redhen_state == REDHEN_STATE_ACTIVE) {
    return FALSE;
  }
}

In addition, all of RedHen's main interfaces are wrapped in theme functions so they can be overridden at the theme level. For example, to alter the default list of contacts, implement theme_redhen_contact_list():

function mytheme_redhen_contact_list($variables) {
  $contacts = $variables['contacts'];
  $header = $variables['header'];
  if (!empty($contacts)) {
    $rows = array();
    foreach ($contacts as $contact) {
      $uri = entity_uri('redhen_contact', $contact);
      $actions = array(
        l(t('edit'), $uri['path'] . '/view/edit', 
        ↪array('query' => drupal_get_destination())),
        l(t('delete'), $uri['path'] . '/view/delete', 
        ↪array('query' => drupal_get_destination())),
      );

      $redhen_contact_type = redhen_contact_type_load($contact->type);
      $rows[] = array(
        'data' => array(
          $redhen_contact_type->label,
          l($contact->first_name, $uri['path']),
          l($contact->last_name, $uri['path']),
          l($contact->email, 'mailto:' . $contact->email),
          format_date($contact->updated, 'short'),
          implode(' | ', $actions)
        )
      );
    }

    $render['table'] = array(
      '#theme' => 'table',
      '#header' => $header,
      '#rows' => $rows
    );
    $render['pager'] = array(
      '#theme' => 'pager',
    );
  }
  else {
    // no results, set a message
    $render['no-result'] = array(
      '#type' => 'markup',
      '#markup' => t('Sorry, there are no contacts 
      ↪that match your criteria.'),
    );
  }

  return render($render);
}

In short, using RedHen's comprehensive APIs, hooks and theme wrappers, a developer can build a complex, elegant solution to meet nearly any use case requiring CRM functionality.

Conclusion

Although we think it's possible to build very robust, large-scale CRM solutions natively in Drupal, ThinkShout's goal in releasing RedHen CRM is not to compete with the enterprise CRM market. Platforms like Salesforce almost inevitably will out-scale any CRM solution that we can build with Drupal.

That said, enterprise CRM solutions often are overkill for small to mid-size organizations and businesses. Moreover, even if your organization does need an enterprise CRM solution, we see RedHen as a natural integration point between your Web site and such a system. RedHen CRM opens the door to the creation of highly innovative front-end CRM tools. We anticipate that collecting and displaying data in RedHen (and Drupal) often will be much more affordable and nimble than trying to develop comparable features upon larger, more cumbersome enterprise packages.

Sean Larkin has more than ten years of diverse leadership and technical consulting experience. He has been working with the best and brightest software engineers and designers in the Drupal community for the past five years. He has led national community organizing initiatives and international relief projects, served as a fundraising strategist for environmental groups worldwide, and ran two open-source software consultancies specializing in Drupal development. He holds a Master of Public Administration (MPA) degree from Syracuse University's Maxwell School. Outside work, Sean is an avid whitewater enthusiast, certified kayaking instructor, raft guide and whitewater videographer. He has been a longtime advocate and supporter of river conservation organizations, such as River Network and Waterkeeper Alliance.

Lev Tsypin has more than 12 years of experience leading technology projects and is a technical architect and co-owner of ThinkShout, Inc., where he leads technical design, user interface and module development. Prior to ThinkShout, Lev ran the highly respected consultancy, Level Online Strategy. In the 1990s, he worked as a consultant with Computer Sciences Corporation (CSC) and Inforte Corporation in Chicago. Before starting Level OS, he served as the Director of Programming at Pop Art, Inc. He holds a Bachelor's degree in Business Administration (BBA) and Political Science from the University of Wisconsin-Madison. When not working to help organizations through technology, Lev likes to take advantage of the areas they protect, be it on foot, skis or bike—at least when he's not wrapped up with his two boys, which isn't very often these days!

LJ Archive