When Written: June 2012
There have been two main pieces of work that I have recently been involved with, each at opposite ends of the web development spectrum. The first was taking an old site where the layout was built many years ago using tables and converting it to a design using CSS. The other project is an information source which for commercial reasons I cannot reveal, but the object is to make this information easily available via web and mobile devices. This second project would give me the opportunity to experiment once again with web services and in particular the new easy development tools promised in Visual Studio. But I was not prepared for the struggle that ensued.
First the story of the old site that needed updating. This site was written and is maintained with Dreamweaver using Templates and Library items. With Templates changes could be made in a single file then updates all the pages based on it, so whilst this made some of the basic changes simple, it was still necessary to go through each web page individually to correct the old code. Search and replace can be used, to help with this, but carefully and thank goodness for backups! But in the end it was one of those jobs that needs someone to go carefully through all the relevant pages editing the HTML and testing in various browsers. This particular site used dynamic pages written in ASP so there were fewer web pages to edit than would seem from browsing the site. There were however still over four hundred pages that needed checking and this took some time. It was certainly a chore, requiring headphones and good music, but it was curiously satisfying removing lots of redundant HTML tags and seeing the tidier and more readable code appear.
With IE9 in quirks mode the nice rounded corners have gone, and trust me the table handling is peculiar as well!
IE9 in it’s standard non-quirks mode behaves like most of the other main browsers now thank goodness.
Obviously as you do this there are times when using tables is a perfectly valid thing to use. For instance when displaying a grid of results then obviously tables are the sensible route. When tackling a task such as this it is very important to test your pages in the various browsers, although thankfully for a lot of designs, the major browsers behave in very similar ways now. One exception to this can be Microsoft’s Internet Explorer. I say can because IE 9 is pretty standards compliant for a lot of things but there is a major ‘gotcha’ to be aware of, and that is ‘quirks’ mode. In this mode the browser will behave as a non-standards compliant older version so that older web sites can still be made to render correctly with modern browsers. This switching between modes can be done by the user or automatically by the browser. The most common way to force IE not to use ‘quirks mode’ is to make sure that on the first line of your web page you have:
<!DOCTYPE = HTML >
With IE you can easily check to see what mode the browser thinks it is using on any particular page by opening up ‘developer tools’ and at the end of the top line of the menu in the developer tools you will be told what the document mode is. By clicking on this you can change it so you can see the differences in rendering in the various modes, you can also make IE9 render as IE 7 or IE8 with this option should you need to test for these, although I personally prefer to use a virtual machine with the actual browsers installed to make sure that all works as it should. The other change we made to this web site was to change the use of Dreamweaver library objects. These are user defined reusable pieces of html that you can select and insert on a web page. The idea is that you only need to change the code in the library object and Dreamweaver will update all the pages that use this library object with the revised code. The drawback to this is that a change to a commonly used library object can result in several hundred web pages being changed and needing to be re-uploaded to the live web site. To make the updating of the library objects that we use for code that might be changed often such as sub-menus on this site, we switched them to using server-side include files which are text files with HTML code in them. This code is then inserted into the web page at run time using the tag:
<!--#include virtual="/my_include_file.shtml" -->
The reason for the shtml extension is to distinguish the include file from a ‘normal’ HTML file. The include file, whilst having HTML code in it, will not be a properly formed HTML file with <head> and <body> tags. To use include files you also need to tell the web server that you wish to do so. In IIS this is a simple checkbox in the Home Directory | Configuration area and in Apache you need to add:
Options +Includes
To your httpd.conf or .htaccess file, you then need to tell it what files to treat as include files with :
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
The advantage of using includes is that should you make a change to the code on one of these files, say for example, altering a link in a menu all you need to do is upload the new file and the hundreds of pages that use it will update immediately, rather than having to upload these pages with their updated library item. There is a little server overhead as it has to process each file to see if it includes this code but even for very busy sites this is not often an issue.
With all these changes done and tested it was time to attack a new project of mine. I can’t currently reveal the exact purpose of this project but the principles involved in its design and development are relevant for similar tasks. The project involves having a database of commercial information sitting on a server out on the web. This data is to be made available via web services so it can be viewed via a variety of mobile devices as well as web applications viewable with a web browser.
My favourite development tool is Microsoft’s Visual Studio which can be used for a variety of languages including JavaScript and jQuery. It also offers me some basic HTML design tools but most importantly it gives me intellisense help with code syntax and full debugging.
Having said all this I do have to agree with Jon Honeyball in his article in issue 214 when he bemoaned the lack of great HTML5 tools that make development as easy as it was in VB3. Remembering back then, I think that the cool thing about developing was that you could drag and drop all your components onto a form and ‘wire them up’ often simply through drop box selections and finally for the clever tweaks you could go to the code and hack away. Whilst writing a nicely crafted product was not often the result it did mean that a program could be written and working in a fairly short time and allowed a lot of people to experience the thrill of doing so. I remember at the time, a very skilled C++ programmer friend decided that he should start to look at writing code for this ‘new-fangled windows thing’ and after two weeks he managed a simple application that opened and closed windows. I had done a similar thing with my copy of VB in about five minutes, so while I admired his efforts, the way Windows coding was going for the majority of us seemed obvious to me. Whilst it is easy to look back with rose-tinted spectacles about the programs we wrote then, such programs would struggle to serve more than a dozen concurrent users whereas now we think nothing of building web applications that have the capability of serving thousands of users. We certainly have moved on but alas a lot of the development tools now are not much more than fancy text editors with code prompting; gone seems to be the drag and drop of objects and the setting of events and methods to get things working. Even where this is still offered you find that the only way to get certain functionality working is to add code to the object involved which if you were not aware what this code might be, can leave you floundering for days. For example, take a look at databinding in Silverlight, you have to manually enter {Binding} tags to the XAML code that defines your page objects. Why can’t I simply navigate from the list control on the phone app that I am developing to my data source that the project knows about, select which fields I want displayed, click OK and have the boring code written for me? Sure Visual Studio does write a lot of code for you when adding webservices for instance but why not the take it one stage further? Is it because the code is so easy that a dyslexic meerkat could write it? I don’t think so. Looking through the volumes written in the forums about the best way to achieve such tasks, not helped that these techniques change with every release of any of the underlying technologies. The examples posted by the obviously very clever people at Microsoft are often unnecessarily complicated and one gets the impression that the contributor is more trying to impress their peers rather than trying to illustrate a technique in the clearest form possible. I wonder if it is just me that feels this way?
Anyway, back to the project …
There are rare occasions when the opposite is true, if I said that to create a RESTful webservice using Data Entity Framework it would take less than five minutes, you might wonder first why would anyone want to do that anyway? But bear with me. Webservices come in two main flavours which describes the way they communicate between the application and the service. These are called SOAP and REST and while SOAP is perhaps better for the financial institutions out there with its better security, RESTful services scale better so it can be argued they are more suited to web applications as they can use the web server’s caching mechanism to improve performance. More to the point in this case a Windows Phone Silverlight application can only consume a REST web service apparently so REST it is.
So with the stopwatch running, let’s see how to create one. The easiest way to create one of these using Microsoft products is first to create a web site in Visual Studio and then to this project you add an item and select ‘ADO .NET data entity’. A wizard will then ask you to select your database or create and empty model. The reason for this second choice is that you can then design your data entity structure and from that a database will be automatically created, clever stuff. If instead you select to create your entities from a database it will then ask you to connect to a data source that your project knows about and then what do you wish to include, normally you just select the tables that you need your web service to access. Once this is done then a graphical representation of the database structure is generated and if necessary you can tweak things from within the UI. The next stage is to create the web service that looks at this data entity, so once again you add an item to your project, this time it is a WCF Service and it is here that the handholding stops. Up to this point the Visual Studio UI has helped you along but now you have to roll your sleeves up and start manually editing code, but at least there are some commented out code hints to help you here. Open up the code file for your web service, if your web service was called mywebsvc and you selected C# as the programming language then this file would be called ‘mywebsvc.cs’ or ‘mywebsvc.vb’ if you chose Visual Basic as your programming language. In this file you need to edit the first line in the Public Class to tell it what data entity to look at. This was the name that was shown in the wizard when you were adding the database to the data entity, in this example it is called ‘MTEntities’, so our altered code line will look like:
Inherits DataService(Of MTEntities)
You then also need to set the access level to your entities, here you can define read or write levels of access but for now we will just allow all areas to have read only access with the command line:
config.SetEntitySetAccessRule("*", EntitySetRights.AllRead)
It would have been a nice touch if when creating this web service that this code could have been auto generated by the wizard, because the next stage is nothing short of magical. Run your web site from within Visual Studio and once the browser has loaded with the default screen add to the end of the url the name of your webservice so it looks something like:
http://localhost:50222/WebSite2/mywebsvc.svc
You will get back a basic structure of your service as an XML feed which should look like:
<?xml version="1.0" encoding="UTF-8" standalone="true"?>
<service xmlns="http://www.w3.org/2007/app" xmlns:app="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom" xml:base="http://localhost:50222/WebSite2/WcfDataService.svc/">
<atom:title>Default</atom:title>
<atom:title>Articles</atom:title>
</collection>
</workspace>
</service>
Doing a query against an OData compliant webservice is a simple as forming the URL
OK, nothing particularly impressive there, as a reward for all that work, but things now start to get interesting. The important part here is the collection href of “Articles” this tells you the name of the dataset that you can query and here comes the magic. You now have a dataset which you can query in a multitude of ways without writing any more code. The syntax to do this follows the OData standard (http://www.odata.org/media/30002/OData%20URL%20Conventions.html ). For a simple example let us assume that our dataset is small so that we are quite happy to return all the records. To do this we just type in the URL:
http://localhost:50222/WebSite2/mywebsvc.svc/Articles
Note here that all the extra bits that you add to this URL will be case sensitive so ‘articles’ would not work. If, as in the real world, we want to return a subset of our data then the URL would look something like this:
http://localhost:50222/WebSite2/mywebsvc.svc/Articles?$filter=Amount gt 100
Entering this would return all the records where the Amount field is greater than 100, or if you only want a few fields returned you could type:
http://localhost:50222/WebSite2/wcfdataservice.svc/Articles?$select=Title,datetodisplay
You will have noticed that the output is in XML which is fine, but what about if you wanted the output from your webservice to be in the more compact Json form? Whilst oData supports the adding of a command at the end of the URL to change the output format WCF Data Services does not support this and you need to jump through some hoops by modifying the HTTP headers in your webservice code. Full support for oData spec is promised but no deadline has been given.
The query language is quite extensive but I’m sure you can see from this little illustration that building a fully queryable web service from a database now really is an easy task, this saves you having to write code in your web service for all the various types of query that yours or others applications might need.
Article by: Mark Newton
Published in: Mark Newton