Where Marketing, Analytics, and UX meet

Drupal 8: RESTful views and jQuery

jquery ajax calling drupal 8 RESTful API

One of the best new features in Drupal 8 is the ability to take views results and easily deliver that data as a JSON object via a RESTful path. In this post I am going to show you how to configure the modules required in order to setup a RESTful API to your Drupal 8 data.

Before we get started, you can click the following link to see my Drupal 8 RESTful Autocomplete experiment. This will be the context of the below explanation for learning the steps to setting up RESTful views in Drupal 8 and consuming them using a front-end JavaScript library like jQuery.

I should say that I have not found a lot of documentation on REST in Drupal, and at times some of the blog posts and technical articles I've found seem out of date, namely because interfaces had changed or tutorials simply didn't work. So while I am certain I will update this article later as I learn more, I thought I would demo what I've found works in Drupal 8 REST to-date.

In this post we will go through...

  • The modules you will need
  • New options for Drupal 8 views
  • The Experiment tutorial

Let's get started.

The modules you will need

Module: RESTful Web Services

Drupal 8 ships with a module called RESTful Web Services. It is not enabled by default, so I recommend you head over to your Drupal 8 sites "Extend" menu and filter for "rest", select the "RESTful Web Services" module and enable it.

enabled the RESTful Web Service module in Drupal 8

Next up, you need to be sure that the permissions are set correctly. To do this, expand the Web Services module information fold-down and click on the Permissions link. Below is my configuration.

Permission for the RESTful Web Service

For the case of the coming experiment (and in most cases) I am fine with anonymous and authenticated users having the permission to make an HTTP GET request for some site content. This is the equivalent of having "read" access to the content on your site. In terms of the other HTTP requests, PATCH, POST, and DELETE, I would reserve these functions for an Administrator or (if you've created the role) a site Contributor. I tend to not let anyone use the DELETE function on the sites I create. Instead I unpublish the content or expire it in a non-destructive manner so that the content does not really go away in an un-auditable manner.

If you understand how Web Services and REST works, then you get the basic idea that a single URL can provide a number of functions for a single piece or group of content. In the olden days of the Web we would have passed a query string parameter that informed some code on the server about how to process the submitted data. For example, if we wanted to delete some content we might have created a page and passed a few parameters on the URL like:

http://enginpost.com/web-service.php?content_type=Post&node_id=12&verb=DELETE

In REST, the expectation is that the same URL is used with different verbs to get the data to the server, and the programmer at the other end reads the verb used and converts over to that action or operation.

Take for example, if you wanted to simply get a drupal post with a node id of "12" a RESTful Web Server call could look like:

GET http://enginpost.com/services/post/12

...and if you wanted to delete it (and had the permission to do so) the call might look like:

DELETE http://enginpost.com/services/post/12

It is worth mentioning that the defaul verb used for URLs is GET.

Here is a quick explanation of the HTTP verbs and how they are used in Drupal 8...

  • GET: The HTTP verb GET is used to request some content, like I said above.
  • POST: If new content is packaged up in some manner (typically XML or JSON or via Query String parameters) and submitted via HTTP using the POST verb, then on the server side one would expect that, with the right permission, content would be parsed and added to a site.
  • PATCH: If new existing content is changed and packaged up in some manner (typically XML or JSON or via Query String parameters) and submitted via HTTP using the PATCH verb, then one would expect the server to update the content.
  • DELETE: Submitting some record identifier via HTTP with the verb DELETE would imply that the server side code should remove the content, assuming the user has permission to remove content.

Pulling back from the abstract, if you were writing your own server-side code to create a RESTful Web Service, you need to write your own routing service that receives all of the page requests, and then parse elements of the requested URL along with the verb used to determine what is being requested. For example, in a PHP controller that handles URL rewriting/routing you might see...

// URL is DELETE http://enginpost.com/services/blogpost/12
$request_parts = explode('/', trim($_SERVER['REQUEST_URI'], '/'));

// $request_parts[0] = "services"
// $request_parts[1] = "blogpost"
// $request_parts[2] = "12"

$controller_class_to_load = ucfirst($request_parts[1]).'_Controller';
// $controller_class_to_load = "Blogpost_Controller"

$request_content = $request_parts[2]
// $request_content = "12"

$http_verb = strtolower($_SERVER['REQUEST_METHOD']);
// $http_verb = "delete"

switch( $http_verb ){
    case "get" :
        // Do something for the GET method
        break;
    case "post" :
        // Do something for the POST method
        break;
    case "patch" :
        // Do something for the PATCH method
        break;
    case "delete" :
        // Do something for the DELETE method
        break;
}

Within the switch statement you can see how we are looking to determine which verb was used in the REQUEST_METHOD and then taking the right action, which might be loading that Blogpost_Controller class and calling it's "delete" function and pass in the $request_content value of 12.  If you were writing this from scratch you would need to do all of that including writing the Blogpost_Controller class to handle the CRUD operations for all Blog Post type content. 

In Drupal 8, Symphony and Drupal take care of all of this for us (Fhew! You don't have to write any of that!) via the RESTful Web Services module. We mostly have to worry about permissions and creating new API services using tools like Views.

Module: REST UI

A module not included with the vanilla install of Drupal 8 is the contributed REST UI module. This module basically takes the rest.services.yml configuration file and sets a number of authentication protocols (like Cookie or Basic), and enables different services data types (like XML or JSON).

Installing the REST UI contributed module

Inside the Extend administrative page, by clicking the Configure link inside the REST UI module's fold-down details you can see how all of this gets configured. Have a look at my configuration.

REST UI data formats and authentication

Remember that the initial RESTful Web Services module is the place we grant roles permissions to these services and verbs. Here we are determining what data type the service returns and how the service will achieve user authentication. 

The key to making these endpoints work within your drupal 8 site (e.g. make ajaxian calls to load content within rendered Drupal site pages) is setting to the right data format and authentication method. As you can see, for any of these verbs, I am requiring "basic authentication" which means the user should at least have started an anonymous session on my website. At the moment the options are either "cookie" or "basic_authentication". Cookie obviously has security issues associated with it (because the permission token information will be persisted as a cookie on the users device) and Basic Authentication only works for a local user with a session.

As well, notice that I have selected the HAL_JSON data type format. At this moment, this is preferential to XML or JSON, specifically if you intend to update content via POST, PATCH or DELETE.

You might also notice that further down that page there are more entities available to "enable" if you want to expose their details through RESTful services.

So how do I authenticate my APIs from a remote native mobile application?

I have no idea yet. My first example here doesn't require that. And neither Basic Authentication nor Cookies seem to be a good candidate for a mobile app authentication method. I imagine there is likely some OATH trick somewhere, but I have no idea at the moment. It will be one of the next things I investigate for Drupal 8 RESTful services.

New options for Drupal 8 views

Now that we have these configurations made we want to check out a few new options in Views. Head over to the "Structure" Administrative menu option and select "Views".

In my experiment I am going to need a couple views: one that returns a list of Blog Post titles and node ids, and one that returns the detailed content for a single Blog Post item. Let's have a look at one example of that.

Drupal 8 view with a display type of rest-export

Let's make note of a few things:

We can now add a display type of REST export

At the top of the view you can now add a display type of REST export, as opposed to Page or Block.

The Path Settings area lets us create our view into a RESTful service

Like a View with a display type of Page, we have the ability to setup a path associated with the display. This is how we configure the end-point for our API.

Advanced Contextual Filters make the service dynamic

In the above example, I included a contextual filter allowing the requestor to pass in a node id to get the details of a single requested node.

Display Format settings area allows for the serialization of an entity

With a few simple selections you can take the entire contents of a node and serialize it (meaning, turn it into a string of data). Earlier, using the REST UI module we specified that the data format of our content GET service would return JSON. At the bottom of the screenshot, in the view preview area, you can see a yellow section showing some details of that resulting JSON string. These are those options in action.

If you visit the above view path on my site and request a particular content node (like http://enginpost.com/experiment/rest/post/64 ) you can see the JSON that now gets returned via my new RESTful api path.

The Experiment tutorial

In my attached experiment, I had two simple goals:

  • As a visitor, I wanted to search for blog posts based on typing sample words into an input box and have a RESTful API return an autocompleting suggestion list so I can see a result list of related blog post titles.
  • As a visitor who selects an autocompleted blog post title, and requests to load the blog post, I want the system to pass the node id associated with selected blog post title into a RESTful API, and load the serialized node dynamically into the experiment page, so I can see the content.

To do that I needed to create two Views with a display type of REST export. One view would return all of the titles of blog posts along with their associated node IDs, while the other would accept a node id via contextual filters and return a single serialized content node.

To implement this ajaxian experiment I decided to use jQuery (most front-end developers are familiar with jQuery, as opposed to backbone or angular) and search for an autocompletion jQuery plugin that used JSON for it's data.

I was happy to find a jQuery plugin aptly named Easy Autocomplete. In fact, all I needed to do was pass the RESTful API end-point to the jQuery plugin and it did the rest of the magic for me. I had to roll my own jQuery ajax to pass a node id and get a serialized node. You can click here to see all of the javascript code for the experiment.

Let's take a look at how I implemented this.

The implementation of the experiment

First, I needed to embed some text and an input box so I could attach the easy-autocomplete script to that field. To do that I created a simple content page with basic text in the page. Then, if you have read my blog post on setting up your Drupal 8 development environment, I was able to look at the source of that page to gleen the right TWIG naming convention so I could add some custom HTML to the page. My new page was node 54, so the TWIG file was named "node--54--full.html.twig. Here is the critical HTML for that template...

// More TWIG markup above this
// removed because I didn't 
// customize it.

<div{{ content_attributes.addClass('node__content', 'clearfix') }}>

    {{ content }}

    // The custom code starts below the content
    {{ attach_library('epft/ep-autocomplete') }}
    <div class="experiment-container">
      <div class="experiment-controls">
        <div><label for='dd_articles_finder'>Find a post:</label></div>
        <input type="text" id="dd_articles_finder">
        <div>
          <input type="button" 
                 value="Load selected blog post" 
                 id="btn_load_selected_article">    
        </div>
      </div>
      <h2 id="article-title"></h2>
      <div id="article-body"></div>
    </div>
  </div>
</article>

If you've been designing or developing with Drupal for a while you know how to add this to your theme and refresh the cache so that Drupal knows about the TWIG template. The first thing I needed to do in the TWIG file was load all of the jQuery, custom JavaScript, and custom CSS into the context of the template. If you read my blog post on adding JavaScript to specific Drupal 8 pages, then you would have noticed the following line in the above TWIG markup...

{{ attach_library('epft/ep-autocomplete') }}

This implies I have a set of resources grouped together under the title "ep-autocomplete" in my themes libraries.yml file. Let's take a look at that section of the epft.libraries.yml file...

ep-autocomplete:
  version: VERSION
  css:
    theme:
      static/css/ep-autocomplete.css: {}
      static/css/easy-autocomplete.min.css: {}
      static/css/easy-autocomplete.themes.min.css: {}
  js:
    static/js/jquery.easy-autocomplete.min.js: {}
    static/js/ep-autocomplete.js: {}
  dependencies:
    - core/jquery

I am including the CSS required for easy-autocomplete, as well as the JavaScript library for easy-autocomplete. In addition to that I have a CSS and a JavaScript file both named ep-autocomplete where I will put my custom CSS and JavaScript associated with the experiment.

So the above TWIG template injects those styles and scripts into the appropriate parts of the rendered page and starts using the new HTML markup elements. Let's take a look at my custom JavaScript to connect the input field to the easy-autocomplete plugin...

Custom JavaScript to attach easy-autocomplete to our input field

(function ($) {

  var article_search_field = "#dd_articles_finder";
  var selected_article = '';

  var options = {
    url: "/experiment/rest/posts",
    getValue: "title",
    list: {
      match: {
        enabled: true
      },
      onSelectItemEvent: function(){
        selected_article = $( article_search_field ).getSelectedItemData().nid;
      }
    }
  };

  $(document).ready(function(){

    $( article_search_field ).easyAutocomplete(options);

  });

})(jQuery);

The options variable is a JavaScript object that tells easy-autocomplete how to function by populating a number of properties:

  • url: The relative URL where it can get the names of blog post articles
  • getValue: This is the name of the JSON data element that will be searched/autocompleted in the input field.
  • list: This is another JavaScript object that has two properties...
  • list.match: If this is enabled it will highlight the element of the title that matches the text typed into the input field.
  • list.onSelectItemEvent: This is an event handler for when the visitor selects one of the autocompleted options.

The event handler onSelectItemEvent tells JavaScript to look at the selected blog post title and go get more data held in the associated JSON loaded from the RESTful api, namely the node id associated with the selected title. It stores that in a local JavaScript variable named "selected_article".

Finally, within the document ready function, I am selecting the input field via jQuery and attaching the easyAutocomplete functionality to it, passing in the configuration options we talked about above.

At this point, if you ran the code it would allow you to type in a word and it would search for that word in all of my blog post titles, returning a list of matches, highlighting the matching portion of the resulting titles. You could select one of the ajaxian results and JavaScript would store the associated node id in a local variable. Not a bad start. The first User Story goal is ready to go! Now, lets look at how to load a selected article dynamically into the current page.

Loading a full selected blog post into the page via our RESTful API

Above we made it so that we could search and select a blog post article. Now lets load that into our page.

At the bottom of the new HTML we added to our TWIG template we create a button to load the selected blog post and two additional HTML elements to hold the dynamic content we want to load from the selected blog post.

      // more HTML above this
      <div>
          // a button to get a selected blog post
          <input type="button" 
                 value="Load selected blog post" 
                 id="btn_load_selected_article">    
        </div>
      </div>
      // the blog post title will go here
      <h2 id="article-title"></h2>
      // the blog post content will go here
      <div id="article-body"></div>
    </div>
  </div>
</article>

 Let's add the JavaScript code to load the selected blog post.

/** previous Javascript above this **/


  $(document).ready(function(){

    $( article_search_field ).easyAutocomplete(options);

    // INSERT the below code inside
    // the document ready function 
    // to add a click event to our button
    $('#btn_load_selected_article').click(function(){
      getArticleDetails( selected_article );
    });

  });

  
  // INSERT the new function here below
  // the document ready function and
  // above the end of the anonymous 
  // self-executing function
  function getArticleDetails( this_article_id ){
    if( selected_article.length < 1 ){
      alert('Make a selection before attempting to load an article');
    }else{
      $.ajax({ url: "/experiment/rest/post/" + this_article_id }).done(function( article_data ) {
        $( '#article-title' ).html( article_data[0].title[0].value );
        $( '#article-body' ).html( article_data[0].body[0].value );
      });
    }
  }

})(jQuery);

Now, when we click the button, assuming that we've already selected one of the blog post titles results, it should call the getArticleDetails function and pass it the node id associated with the selected blog post title.

When the data comes back from the RESTful API (remember that this is an asychronous function call, so you may have to wait a moment) we look into the JSON data and load it into the HTML of the available HTML elements we created earlier.

What about updating nodes via RESTful APIs and jQuery

This is, by far, the most complicated bit and requires that the current user have the right permissions and is currently associated with a website session within the Drupal 8 site. 

As we said before, to PUT a change to a node required that a user have the Contribute or Administrator role. If they do not, then they will not be able to update the blog post.

Let's imagine adding a goal to our experiment...

  • As a contributing user I want to be able to submit a change to a blog post title using the REST HTTP PATCH verb, so I can update the blog post title using ajax.

The included Drupal 8 RESTful Web Service module already includes an interface for allowing us to PATCH a change to a node. In fact, when we were looking at the REST UI module's interface it showed it directly on the screen. We need to call the PATCH verb on an ajax call to the URL  /node/<node id> while including a specific drupal patch string along with a session token, and then assuming we have permission, we should be able to update the node.

Wait! Sound complicated? Well, it is and it isn't.

First, let's assume we created a fairly straight-forward way to let the user input a new title into an input field with an id of "blog_post_new_title_field" along with a new button to attempt to save the new title to the existing node. 

<input type="text" id="blog_post_new_title_field">
<input type="button" id="new_title_submit" value="Update the title">

Let's take a look at what javascript we would need to write to get a session token and submit the change and update the blog post title.

(function ($){

    // At the top of our ep-autocomplete.js file
    // we are going to INSERT a new variable to
    // hold the session_token
  var session_token = '';

    // Next we will INSERT a function to get
    // the session token value from Drupal.
    // Drupal has a RESTful service for this.
  function getToken(){
    $.ajax({ url: "/rest/session/token" }).done(function( data ) {
      session_token = data;
    });
  };

    /** previous JavaScript here **/

  $(document).ready(function(){

        // Now we need to INSERT a call to
        // the getToken function inside
        // our document ready function.
    getToken();   

        // And we need to INSERT code to
        // connect our new button so it 
        // can try to update the blog post.
    $('#new_title_submit').click(function(){

      var patch_string = '{"_links":{"type":{"href":' +
                         '"http://enginpost.com/rest/type/node/post"}},' +
                         '"title":[{"value":"' + 
                         $("blog_post_new_title_field").val() + 
                         '"}], "type": [{"target_id":"post"}]}';

      $.ajax({
        url : 'http://enginpost.com/node/' + selected_article,
        method : 'PATCH',
        data : patch_string,
        headers : {
          'X-CSRF-Token' : session_token,
          'Accept' : 'application/json',
          'Content-Type' : 'application/hal+json'
        },
        success : function(response, textStatus, jqXhr) {
          getArticleDetails( selected_article );
        },
        error : function(jqXHR, textStatus, errorThrown) {
          // log the error to the console
          console.log("The following error occurred: " + textStatus, errorThrown);
          alert('There was a problem saving your changes.');
          getArticleDetails( selected_article );
        },
        complete : function() {
          // Whether the call is successful or fails
          // this code will always run at the end.
        }
      });

        /** more previous JavaScript here **/

In this case I am using jQuery ajax to call the Drupal 8 RESTful API to update a node. Notice that I simply append the ID of the node I want to update. The actual update information is passed in the patch_string variable. What an insane serialized string. Let's have a closer look...

{
    "_links":
        {"type":
            {"href" : "http://enginpost.com/rest/type/node/post" }
    },
    "title":[
        {"value": "This is the new title"}
    ], 
    "type": [
        {"target_id":"post"}
    ]
}

This is a JavaScript serialized object using the HAL+JSON format. I am not completely certain of what all of the different items mean, but it seems that it is doing the following.

If we assigned this to a variable named "patch_object" then if we wanted to see the type id of the serialized object we were updating we might type...

patch_object.type[0].target_id

// RETURNS: "post"

And if we wanted to see what the canonical path to the RESTful service associated with that content type id might be (I am guessing that is what this means) we might type...

patch_object._links.type.href

// RETURNS: http://enginpost.com/rest/type/node/post

Beyond the two "_links" and "type" properties any field that seems to be changing gets it's own property in the patch_object. So, for example, if we are changing the title field of the blog post we could look at the change by typing...

patch_object.title[0].value

// RETURNS: This is the new title

So that is the sensibility of the patch object serialized.

In the above JavaScript we are first trying to make an asychronous call for the session token. Remember that if I want to PATCH and update or POST a new piece of content to be saved I need a token. This way Drupal (and Symphony) can check to see if the user associated with the session has the permission to do those things. This is achieved by adding headers to the HTTP request in order to make that happen. Notice there are two other required headers I have to set in order to get Drupal to give me a result.

  • Accept: This is the header that tells Drupal 8 on the server that I am expecting to get a response formatted as JSON.
  • Content-Type: Remember that we are sending data, and so this tells Drupal 8 that the patch string conforms to HAL+JSON format (this is where we specify the _links object embedded in the patch object).

Next I pass the patch string as data, which appends that serialized object to the RESTful request.

This is all asynchronous and if the ajax call is successful it will call the jQuery ajax success function. The same is true for handling an error response. In the case of my example, if it is successful I then request a fresh copy of the blog post using our API (the result should contain the updated title).

Go back and check out the experiment to see all of this in action.

 

Add new comment

*

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
By submitting this form, you accept the Mollom privacy policy.