Web Pages that kinda work

We started this journey seriously in 1997 (time-line and site history) having played around with a web presence since 1995. We slavishly tried to make our web pages work exactly the same way on every browser. We had only about 150 pages on the site and there were really only two browsers in those days - Netscape 4.x and MSIE both of which were wildly different. We were able to write cross browser pages that displayed well on both major families by keeping things simple. Then in 1999 we decided - then - to push the envelope - we decided that page navigation was important and that pop-up/pop-down/fly-out menus were the best way to simplify user navigation. Further, as the site grew, CSS would become increasingly important. Those two apparently simple decisions - which we still consider to be right for big sites - really got us into the cross browser business and forced us to make some technical decisions - trade-offs - that have consequences for certain users of this site.

In 2007 we stood back to look at what has happened since those early decisions. Can we get to a single page content that will render acceptably on at least the browsers we care about, a single standards compliant'ish web?

If we dropped pop-out menus, removed the right hand menus and were prepared to live with some horrible font size rendering differences - then the answer would be Yes. In fact this is what we did from 1995 to 1999.

As it is we are incrementally removing differences - but very slowly. The old NS4.x differences are gone - we now ignore this browser. With MSIE 7 more differences are going but are being replaced with new challenges over delivering usable context to small screens. We suspect it will be another 5 or so years before we can use a single page format. And during the next 5 years, in this business, anything can, and usually does, happen.

Our Priorities

We consider our priorities for this site to be:

1. Site maintenance and publication speed: with over 4,500 pages - still growing - we need to be able to build and change pages and layouts fast. We also want to be able to change site and page design quickly. For us this implies extensive use of CSS and because CSS is inconsistently implemented, though the inconsistencies are declining, we have a cross browser problem. In 2003 we moved to a CSS liquid layout which was surprisingly simple but had the effect of amplifying the cross browser problem.

2. User Experience: We consider the major useablity factor for big sites to be the ease with which visitors can get to the information. For us, this meant left and right menus and pop-up/pop-down/flyout menus. In 1999 when we built our first pop-up menus they were like hens teeth - pretty rare. Today (mid 2006) they are increasingly the norm. Pop-ups originally got us into javascript which really got us into the cross browser compatibility business. Since 2003 we have been progressively replacing the javascript pop-ups with pure CSS pop-ups/pop-down menus. And in mid 2006 we started replacing our MSIE pop-out Javascript with MSIE 5/6/7 CSS menus. And a lot of fun that was.

   We have been thinking a lot about whether there is a role for Web 2.0 and/or AJAX techniques on the site. Our musings to date.

3. Visual Appearance: It might not show but we do consider the site look to be important! This encompasses both the aesthetic look and feel of the site and particularly how it is laid out on the increasing range of screens sizes - hence our move to liquid layouts back in 2002. Again CSS is the dominant component of this aspect of site design. Again cross-browser compatibility the consequence.

Our Technical Choices

We made the following choices:

1. We follow the W3C standards for HTML 4.01 and CSS2. We support browsers that support these standards even if there is no commercial justification for doing so. Frankly Gecko/Opera do not have enough market share between them to make it a sensible decision - nevertheless that's what we do (Ed originally written in 2003 - our current access stats show 39% generic Mozilla, 23% Firefox and 21% MSIE). Generally we would say that our site is optimised for this class of browser. We do not (we cannot) ignore MSIE so we spend slightly less time but do provide equivalent features. With MSIE 7 promising good (W3C/Gecko like) CSS support (in mid 2006 it was a promise) we may be able to move this browser into our optimised class.

2. We are decreasing as rapidly as practical our use of javascript. We see it as both a major source of cross-browser compatibility problems - the DOM and AJAX do not really change this - and an ever increasing source of security related problems. We cannot, currently, remove Javascript entirely. We have about 30 or so lines for each page - more for MSIE. There is a place for javascript but, in our view, page layout effects are not that place. We would love to see an extension to the pseudo-class selectors in CSS to allow for click triggered events. With :rclick and :lclick or similar selectors we could do all kinds of neat Web 2.0 type things and we would be left with 10 lines or so of benign, but essential, Javscript.

   In mid 2007 we started looking at what, so called, Web 2.0 interfaces could do for our style of pages.

3. We are not - at this time - using XHTML. We have an number of XHTML pages on the site,and they are growing slowly, but we cannot see any compelling arguments for a full conversion to XHTML site wide. XHTML currently offers no features that we need. All pain for no apparent gain is not something we enjoy. HTML 4.01 is still good enough. (Ed: Orginally written in 2003. It now (2008) appears that W3C also thinks that XHTML is dead. With HTML 5.x being the way forward. We are dumping our XHTML pages as quickly as possible but we now have a problem with all those artificial closers for img and <br /> tags and the like that we were once promised were the safe way forward for ever as we moved into the bright shiny world of tomorrow. Well if our pages no longer validate properly that is just too, too bad. When the standards guys blow it - that's it. Game over. No more credibility.)

4. Other browsers - those which are neither W3C or MSIE compatible are increasingly becoming a problem since they comprise three distinct groups.

   1. Text based browsers. Mostly, but not exclusively, used for accessibility reasons. There are, however, a surprising number of folks with slow connections or who dislike the glitziness of the current web who also use them. This group must not be ignored.

   2. Older, oddball or cloaked browsers. Here we care much less.

   3. Mobile devices. The functionality of this group is rapidly increasing but with their generally small screens they represent a major obstacle to right and left hand menu design (not enough space) and tend to have javascript and CSS support ranging from non-existent, through pathetic, to full blooded W3C (mini-Opera etc).

   We have no desire to replace one set of differences with another to cover this wide range of default users. We are trying to create a single page strategy that will work for all of them equally well (or should that be equally badly) and are increasingly moving toward a strategy of omitting all javascript and removing right hand menus - because the screen size is too small to support them and many browsers in this class also have problems with CSS right positioning. We also display a message indicating that the browser is getting - in our view - a less than optimal browsing experience - which is probably unbearably annoying and outrageously arrogant.

Our Implementation Choices

We made the following choices in 1999 which still largely govern our site organisation:

1. We do server side browser sniffing and generate appropriate page content. Depending on your browser the CSS and HTML may be different. Currently we have three versions - W3C compliant, MSIE compatible and a default version. With the advent of MSIE 7 we see a future where we could handle MSIE 7, Gecko and Opera in a single page type and will use (from mid 2006) MSIE's conditional comment feature to handle MSIE 5/6/7 differences. Thus our plan is to continue to use server side sniffing for two page types - Gecko/Opera/MSIE and the rest.

2. We do not care about consistency between browsers. If you look at this page in both Gecko (Mozilla et al) and MSIE you will see different page banners. It would have taken many hours to get them to look exactly the same due to CSS quirks. In our view it does not matter. The look is consistent within the browser it does not need to be consistent between browsers. This approach has saved us many, many hours of work for what we consider would have been a trivial return.

3. We are increasingly removing Javascript for page layout effects whenever we can. We have a series of wireless calculators and a trivial regular expression tester which are written in javascript and think functions like this, as well as field validation, auto-completion, etc., are good uses of javascript - page layout is, IOHO, an abuse of Javascript - though it must be said at the same time that we are looking at exploiting some AJAX techniques which it can be argued run counter to our declared strategy. CSS should - and does in good implementations - provide all the page layout effects we currently require.

4. We worked hard to provide pop-up/pop-down, multi-layer menus for site navigation. But our motive for pop-ups is to simplify the users navigation task - and to give a very quick spatial tour of the site as a side benefit - not provide a unique method to get to a page. If a user's browser cannot or does not support pop-ups they can still get to any page via multiple page clicks. We would argue that our navigation degrades relatively gracefully if CSS or Javascript support is not present. We think that any other navigation method is irresponsible and plain bad design.

5. We have used right hand menus since 2002. They never contain essential page navigation. Rather they are provided as a convenience, typically containing resource links and perhaps some context based short-cuts. Eliminating them entirely for certain classes of browsers is not catastrophic - site navigation is still possible. We typically send the menu bar to all users but with a CSS property of hidden/invisible for selected browser classes. Thus if a browser is capable of making sense of them (such a text browser) it can do so - else it's an overhead - albeit a somewhat painful one for limited bandwidth users such as those with mobile devices.

6. Our pages are rigidly structured into top banner, central content, left menu, right menu and footer and we make extensive use of Apache's XSSI feature to simplify page structure and especially to enable fast site-wide changes. One of the side-effects of XSSI is that javscript and CSS is contained in the page and does not use <link rel= constructs so speeding page-load as well as minimizing server load. We abandoned graphic roll-over effects in 2001 in favor of CSS versions and have never regretted it. Between Text color, background and borders the CSS effects provide for endless hours of fun.

Implementation Consequences

There are consequences for our implementation choices:

1. USER_AGENT Strings: If a browser does not generate a sensible USER_AGENT string it can get a default page. There are no official USER_AGENT standards - we have written to the W3C on this topic with no response to date - but there is a well established ad-hoc standard. If a browser does not follow the ad-hoc standard or users decide they wish to experiment with the browser string then they will get a default page. If the purpose of the experimentation is to defeat browser sniffing they will succeed - now what? A pointless exercise and IOHO the closest some folks come to creativity - after breathing at birth that is. Such users will still be able to get to the final page - though they will require more clicks - a perfect reward in our view.

2. Cached pages: We have had over the years a number of complaints that poorly designed web caches which only use the target URL generate an incorrect page view. For example, if a MSIE user's access caused a remote web cache to be filled then a subsequent Gecko user's request, which was satisfied from the same web cache, would deliver an MSIE formatted page. Generally this way round is harmless - such is commercial power of MSIE - but the other way round can be quite disastrous.

3. Disabling CSS: Users can disable CSS in most browsers. We have no idea why a user would do that since there are no security issues involved - unlike javascript which we understand all too well, hence our decision to reduce/eliminate the use of javascript.

Outstanding Issues

We have a long way to go - this is part of our to do list:

1. Accessibility: We think we should do an overhaul of CSS definitions to improve Accessibility. We have too many px and font-size directives in the CSS and think we need to rationalise the situation. Update We did a brief check at the end of 2005 on accessibility and were very disappointed with the very small amount of good advice out there on this important topic. Tons of motherhood - but a little short on practical advice. As always there are exceptions such as webaim.org and the W3C WAI.

2. Pop-Outs: We will be adding more depth and more selection in our pop-outs to provide faster access to data - probably going to four or even five-level pop-outs. We will only do this for CSS based menus.

3. Small Screens: We think we should do something about providing access to the increasing number of smaller screens - mostly mobile/PDA/Telephone thingies. Most Liquid designs handle a range of big screens well but are hopeless for smaller screens - ours is no exception. Paradoxically we suspect that the solution to this problem is do more work on our default page format, that is, provide no features. But it also has serious implications - we think - for right/left menu usage - the space is not there for dual menus. Good site search features may be the best way forward for this class of user.

4. Languages: Next year (2006) we will be starting to provide some of our pages in multiple languages. We want this to be transparent to all users. We will use Apache's multiview features to provide this based on the browsers language priorities.

Web 2.0 - Theory and Practice

So just what is Web 2.0? If the term Web 2.0 is meant to describe a rich user experience then we would argue - through the use of pop-up/down menus and deep linking - we have been using Web 2.0 techniques since 1999. If you limit it to AJAX (javascript) techniques then we have been going in the opposite direction!

Some thoughts about our site and its pages:

1. We provide a lot of text based pages, we use a minimal number of images, few and very simple forms and currently no audio or multi-media content - though this may change over the next year or so. We are perhaps not ideal AJAX users in this respect.

2. We use a lot of links on our pages. Both internal and external. We have been thinking about using AJAX to suck in site based linked page content into an overlay div - rather than require a full page load. We currently have a simple visual indicator that a link will open a new page (grey background) or not. The new scheme would require another visual clue. Alternatively we could add the linked pages to the page content and use a trivial js function make it visible.

3. It's always tough when writing explanatory material of any kind to decide what subsidiary material should be added to emphasize points or provide background explantions. We tend to always add supporting material because we feel that the moment the reader is lost the rest of the material is, frankly, useless. Even the positioning of such material can be a difficult decision to avoid breaking reader concentration with irrelevancies - especially for experienced readers who have familiarity with the subject matter. We have been experimenting with hidden notes and using javascript and/or css to pop-up the additional material. This also involves a different style of writing - so the challenge is as much literary as technical.

4. We have never been fans of glitz. While we can look at the iMac and its bubble graphic menu effects and be awe struck - our second thought is - what else could we do with that CPU power? The only time such feature are useful is to allow a lot of, essentially unreadable material, to be packed into a very small space, but trivial CSS techniques can be used to expand them on :hover events.

5. We have never used photographic or movie based material - even in places where it would be very useful. The reasons are historic. Such techniques were either not available and/or viable when we started. That has now changed. We need to start experimenting with this kind of material.

Some thoughts on AJAX, Javascript libraries and related issues:

1. Size: The size of most javascript libraries offends us. The smallest we could find is 70K and the biggest weighs in at over 200K. Please. These libaries should contain tools to analyse page use and generate minimal js files or as a minimum section the libray for content such as base, form validation, graphics, effects etc..

2. security: The thought of random XMLHttpRequest requests hitting our server makes us wince a little. We need to think this one through a tad before implementing a live version.

Our Conclusions

1. If we used a simple page model without pop-ups and without CSS (or minimal CSS) we could probably write single pages that would display on almost all browser environments and avoid any browser sniffing. In our view such a strategy is not viable for today's web.

2. From our web statistics we judge that we provide a good browsing experience to about 97% of our human visitors. Its not as good as we want but perhaps that's as good as it gets.

We encourage comments on our approach, differing implementation experiences or alternate strategies and trade-offs. We're not interested in rants, diatribes or stream-of-consciousness stuff. But welcome comment from grunts, like us, who live and work in a practical world, solving practical problems with finite resources available and genuinely trying to provide the best experience they can for site visitors.

---

Problems, comments, suggestions, corrections (including broken links) or something to add? Please take the time from a busy life to 'mail us' (at top of screen), the webmaster (below) or info-support at zytrax. You will have a warm inner glow for the rest of the day.