Is the World Wide Web a Shovel or a Hole?

Peter Outteridge
Internet Information Services
Consultant to CSIRO Division of Information Technology

CSIRO Division of Information Technology
Building E6B Macquarie University
Locked Bag 17
North Ryde NSW 2113
Australia

Phone: +61 2 325-3102
Fax: +61 2 325-3101
E-mail: <peter.outteridge@dit.csiro.au>
URL: <http://seweb.dit.csiro.au>

Abstract
Software is now a major contributor to Australia's export income, as well as being a vital component in developing and maintaining the competitive advantage of Australian businesses, both domestically and overseas.

This paper describes the planning and implementation of the SEweb project. Although SEweb is not a technically complex project, it provides an interesting insight into how a new, and rapidly evolving, technology can be used. Outcomes were difficult to accurately predict, and the range of stakeholders ensured that many diverse viewpoints had to be assembled and, ultimately, accommodated.

The aims of SEweb, its audience, information content, and technology infrastructure are described. In particular, the paper describes the pitfalls and lessons learned. It will be of interest to organisations considering using the Internet to support their business processes.

Keywords
Internet, World Wide Web, software engineering.

Contents

  1. Is The World Wide Web A Shovel Or A Hole?
    1. Abstract
    2. Keywords
  2. Contents
  3. Introduction
    1. The Problem of the Shovel
    2. Software Engineering in Australia
      1. Definition of Software Engineering
    3. The Internet
      1. The Internet as a Shovel
    4. The Origins of SEweb
      1. The Aims of SEweb
  4. Outline of the Project
    1. Project Components
      1. Technology Infrastructure
        1. Authoring Environment
        2. Publishing Environment
    2. Content
      1. Content Providers
    3. Deployment
      1. Industry Partners
  5. Outcomes
    1. Future Plans
    2. Lessons
      1. Lack of Tools Hinders Development
      2. Dynamic Web Services are on the Move
      3. Content is Vital
      4. The Benefits of the Distributed Model
      5. Linking Resources to People and Organisations
      6. Intellectual Property Issues
      7. Point-Of-View
  6. Conclusion
    1. References
    2. Acknowledgments
    3. Author Information

Introduction

The Problem of the Shovel

There's an old saying that people buy holes not shovels. People sometimes buy products for their own sake rather than because there's a real need for the results ownership can bring. The end result of the purchase is just a bonus, and might not even be needed at all.

Vendors encourage us to buy their products by selling features and benefits, leaving it up to the buyer to work out if they really need the benefits. Sometimes there are no benefits, and often the benefits only happen if the buyer adds something else, usually at a cost in time, effort or money.

And so it is with the shovel. Vendors sell them and consumers buy them, believing it will help improve their lot in life, and besides, everyone else has one. Shovels make great holes, but only with some hard work and effort. The holes are undoubtedly useful, but only if they areactually needed.

Could it be that the World Wide Web (the Web) is like the shovel? Is there really a need for the holes? Is it understood what needs to be added to the shovel to get something useful. Right now, are the kinds of hole the shovel can dig really useful?

Software Engineering in Australia

The Australian information technology (IT) industry contributes significantly to the Country's exports, and revenues are on par with the major agricultural and mineral exports (Fra95). Further, IT now plays a vital role in supporting efficient and effective industry, commerce and government in Australia. Since software development and deployment are now occurring in almost every industry, as well as at all levels of government, it is very difficult to assess the total expenditure in these areas. It is clearly a substantial amount, and its leveraged value is likely to be much higher. As an example of the cost of software in Australia, the value of legacy systems alone is believed to be around $30B.

Thus, improving the performance and quality of this sector must have wide-ranging and worthwhile benefits to the Country, both domestically and internationally.

There is little universal acceptance of the need for, nor the relevance of, much of the discipline of software engineering within the software development community. To some, it is seen as the domain of the academic, replete with ponderous and restrictive methodologies and languages. To others it is seen as useful background, but barely relevant to the day-to-day task of actually developing working, commercial software. There is, however, growing opinion that software engineering is maturing and that it is now capable of providing a solid, engineering approach to the design and construction of software systems.

For those interested in learning about, using and nurturing skills based on software engineering principles, there is much to be done. Unless one is a graduate of a computer-orientated course with software engineering content, a great deal of hard work is required to build a comprehensive set of knowledge and skills. No matter how the knowledge is acquired, maintenance of this knowledge is an ongoing task, with the half-life of IT knowledge estimated at only two years.

Definition of Software Engineering

The underpinning of much of the software development effort is the discipline of Software Engineering (SE). This is defined by the IEEE (IEEE94) as:

(1) The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software.

(2) The study of approaches as in (1).

Peter Horan (Hor95) from Deakin University suggests a more commercial view, based on an original definition by Blum (Blu92):

The application of tools, methods and disciplines in a cost effective way to produce and maintain solutions to a real-world processing problem automated partly or wholly by software.

The Internet

The history and nature of the Internet are very comprehensively described elsewhere (RFC1206, RFC1207, RFC1462) and the reader probably needs no introduction to its capabilities, both good and bad. A major characteristic of particular relevance here is that it is growing at a phenomenal rate, and is fast becoming ubiquitous. The quantity and range of information resources, and the ease with which they can (nearly always) be located, is staggering. Equally, the number of people using the Internet has increased exponentially, providing a potentially large consumer market.

As a platform for making material widely and easily available, the Internet is already a very valuable facility. The searching and browsing tools available through the Web already make it relatively easy to use, such that even beginners can locate and access worthwhile information resources. Every day these capabilities improve, as does the variety and richness of the resources.

Of course, this is not to suggest that there are no drawbacks to the Internet. It can be slow, it can be unreliable, and it can be frustrating in its reluctance to divulge material located easily only the day before.

The foregoing comments indicate that the Internet might have great potential for business and commerce. Indeed, this is a widely held view amongst journalists and industry watchers. The Internet appears to be a potentially useful tool for the development and promulgation of data, information and resources to a homogeneous group.

The Internet as a Shovel

But caution is required. Are buyers only looking at the shiny new shovel, one with a skilfully shaped blade and nicely polished handle? Can this shovel really dig holes, and if it can, are they the sort that are needed; not just caverns into which the hapless buyer might simply disappear? Do people really understand just how much hard work is needed to plan and dig the holes that ultimately might benefit the enterprise?

Today there are no easy answers to these questions. The sponsors of this project recognise this, and so the project discussed in this paper is a pilot, a proof of concept. The aim is to discover what kind of holes can be dug with this new tool, and what effort must be expended to gain benefit.

The Origins of SEweb

The Federal Department of Industry, Science and Technology (DIST) and CSIRO share a strategic interest in the Internet.

DIST is concerned with improving Australia's competitiveness and efficiency, particularly for small to medium enterprises (SMEs), many of which are operating below critical mass. The AusIndustry Enterprise Network Programme places `brokers' in organisations to identify common interests and introduce organisations to each other, with the aim of creating critical masses. DIST's view is that this brokering process might also be undertaken effectively using the Internet.

CSIRO's interest is similar, but directed more at the individual enterprise and how it might use the Internet as a tool for commerce. In 1994 it appointed a full-time manager whose role is to assist industry in understanding and evaluating the Internet as tool for commerce.

CSIRO, several universities and others had been involved in moves during 1994 to establish a Cooperative Research Centre for Software Engineering. Unfortunately the CRC did not receive funding during that round of grants. However, the process did indicate that there is a strong desire within the software engineering community, both academic and industry, to work together in the interests of improving software engineering practice in Australia.

In early 1995, DIST and CSIRO established a project to examine ways in which the Web, and other Internet-based tools, could be used to support the activities of a vertical market sector. This was made possible through a grant from DIST.

The software engineering community was chosen for this pilot project for several reasons. Firstly, it is likely to be amenable to a technology-based service, and would be comfortable using an online service. Secondly, it already has a reasonable level of Internet connections. Thirdly, software engineering had been identified by DIST and CSIRO as an area for priority funding.

The project involves industry, government and universities, but with a major emphasis on supporting software development activities within Australian industry. The project provides both content and communication, using Internet tools such as the Web.

The proof of concept project became known as SEweb, or the Software Engineering Extension Service.

The Aims of SEweb

SEweb's overall aim is to improve Australia's performance and competitiveness in software engineering. This could result in improved software systems, lower development times, increased robustness and maintainability, lower costs and higher customer satisfaction. Australian industry and government will benefit, and Australian software products and services will be more attractive to overseas buyers.

The stated aims of SEweb are:

  1. To enhance the competitiveness of Australian industry by encouraging the use of world's best practice by the Software Engineering community;
  2. To provide a vehicle for technology diffusion between researchers and industry on matters relating to Software Engineering;
  3. To carry out the role of an electronic `broker', where organisations can discover potential new business partners and opportunities; and
  4. To become the definitive source on matters relating to Software Engineering in Australia.

Outline of the Project

Project Components

In this paper, the SEweb project is discussed in three parts:

Technology Infrastructure

The underlying infrastructure for SEweb consists of hardware and software to support authoring and publishing environments. The underlying technology supporting SEweb is not the main theme of this paper, and is dealt with in a fairly superficial way. The reader is invited to contact the author if further information is required.

World Wide Web electronic publishing involves two main elements, authoring and publishing. Authoring is to development and organisation of content, and its incorporation into a suitably structured set of Web pages. This involves the definition of content, reformatting and converting documents, Web page design and structuring, and graphics and scripting support work. Publishing is the deployment of pages and server functions on the Internet. This requires appropriate tools to allow users to locate, browse and download material. Information must be organised appropriately, held securely if required, and access must be controlled and logged.

Authoring Environment

Whilst the selection and deployment of the publishing infrastructure was relatively simple, the choices for authoring were daunting. Tools for Macintosh(TM), Unix(TM) and Windows(TM) were reviewed, but none of these environments emerged as obviously superior.

The Macintosh was eventually chosen as the main development platform, in the main due to the author's preference for this system. Several HTML markup tools were trialed, with the commercial text editor BBEdit(TM) emerging as the most effective when combined with Lindsay Davies' excellent HTML Tools package. The Macintosh environment also offers a wide range of good quality tools for developing and manipulating graphics.

The functionality and usefulness of Web services can be dramatically enhanced by scripts to control the content and presentation of pages. Scripting tools for SEweb required little consideration as Perl has become the de facto language for Web scripting under Unix. A Macintosh port of Perl proved useful for some early scripting activities.

During this process it became clear that, whilst there were many tools available to markup text, there was almost nothing to assist with the overall management of the Web site. Individual tools exist to markup text, convert graphics, check URLs, analyse logs and so on. However, there appears to be nothing yet that hides most of the technical detail from the Web developer. No doubt these will appear in time.

Publishing Environment

At the start of the project a decision was made to develop and deploy the supporting computer systems and Web technology for SEweb at an early stage. This would allow demonstrations to be conducted and would provide a basis for comment and criticism. It also provided an opportunity for the project team to investigate current Web technology and practice, and to settle on standards for look-and-feel.

Within the Sydney Laboratory of CSIRO's Division of Information Technology, there was a significant pool of expertise in setting up and maintaining Web servers. CSIRO already had a number of Web servers spread geographically across Australia, and was developing advanced Web-based services and applications. The establishment of a new server was, therefore, a fairly straightforward process. Without this support, the establishment of a server could have been much more difficult.

The SEweb service is based on the CERN HTTP 3.0 server running on a Sun SPARCstation(TM) with SunOS 4.1.1.

Content

Very early in planning for SEweb, it became clear that content would be a key factor in its usefulness and acceptance. Content covers a wide range of material and, in this context, includes educational, technical, reference, and summary material.

A wide range of material will be available on SEweb, including:

Content Providers

Many Web sites are managed by individuals or organisations and present a proprietary view of a particular subject area. In addition, they may seek to be self-contained as far as content is concerned, which is reasonable for vendors and archive sites. With SEweb, it is important to avoid proprietary bias, and to minimise domination of the service by individual views. There are also insufficient resources within the SEweb project team to adequately provide a comprehensive range of material, particularly in the short time available for a pilot.

SEweb had always been seen as a collaborative project, with many partners working together for the benefit of the software engineering community. The academic community would, of course, be a major source of content. This community is used to collaborative activities, and generally likes to know what is happening at other centres working in similar areas. This provides a very effective environment for the development of high-quality material. It was therefore logical that this collaborative model be used in the provision of content.

Some research was undertaken on the activities and capabilities of Australian universities and research organisations (both federal and industry) working in the software engineering domain. It became clear that there was sufficient breadth to provide good coverage of the software engineering discipline. To break up the subject into manageable pieces, the project team decided to nominate a series of subjects, and to appoint a sub-editor to manage each subject area. Whilst some guidelines were provided to sub-editors, the organisation and coverage of their area are very much left up to them.

A series of meetings with interested parties were held with organisations identified during the research. As a result, a number of universities and research organisations were appointed as sub-editors. These include:

Deployment

The deployment phase of SEweb, at August 1995, is just beginning. Armed with a skeletal service, industry and government organisations are being asked to participate in evaluating and developing the prototype service.

The prototyping nature of this project is important, and it is planned to rapidly refine the service as feedback is received.

Currently, site visits are underway to industry partners who have indicated an interest in participating in this pilot project. The aim will be to explain the objectives of SEweb, to demonstrate the server facilities and content, and to seek agreement on a plan for review and comment by the partner. It will be emphasised that this is a pilot project (ie, a proof of concept) rather than a fully fledged service. The comments from industry partners will be pivotal in determining how the service develops after the pilot phase.

Industry Partners

Amongst the companies that have indicated a desire to be involved in the SEweb project are: Other commercial and government organisations are likely to become involved, but the service will not be made generally available until after the pilot stage.

It is expected that some of the industry partners will also make material available on SEweb. This might be through their own home pages, or as part of a moderated subject area.

Outcomes

Future Plans

The pilot SEweb project is due for completion at the end of 1995. At that time, the progress and benefits of the service will be determined and a series of recommendations made to DIST. Broadly speaking, there are three possible outcomes: In the short term, a service such as SEweb can exist on an ad hoc basis, but to build quality, depth and continuity, some form of content plan is essential. One option to meet this need is to establish an editorial board, with representatives from both the academic and industry partners, that would oversee the organisation and overall quality of content.

Lessons

Although the SEweb project is far from complete, a number of lessons have already become clear. Amongst these are the following, presented in chronological order rather than order of importance.

Lack of Tools Hinders Development

Currently there are few really robust tools to support the total Web development activity. The are good editors, graphics tools, servers, clients and helper applications. But they are not really integrated, particularly at the authoring and publishing level. An object-based repository that manages pages as collections of objects, and shields the author from HTML and low level scripting is certainly desirable.

Dynamic Web Services are on the Move

Much of the material on the Web today is organised into `static' pages, linked through fixed links (URLs). This structure has worked sufficiently well during the development of the Web, but it has created a maintenance and logistics nightmare. The trend on the Web is to `dynamic' pages, constructed in response to a set of conditions, currently provided by scripts written in languages such as Perl and C. The more interesting sites already use extensive scripting, and the arrival of multimedia enhancements such as HotJava and QuickTime will only accelerate this trend.

This has implications for those building and maintaining Web sites. Complexity will increase and new, scarcer skills will be required.

Content is Vital

The Web is often criticised for its low signal-to-noise ratio. It contains enormous amounts of material but large parts of this are of low quality and limited usefulness. The real problem is, however, one of discriminating between good, bad and mediocre.

Whilst SEweb is not immune from this problem, the concept of sub-editors and an editorial board goes a long way towards ensuring material of consistent and acceptable quality.

The Benefits of the Distributed Model

The Web is perfect for collaborative and distributed projects. Material and expertise do not have to be concentrated in space, but rather can be distributed at centres of excellence and the whole operated as a virtual library. This characteristic of the Web is the keystone of SEweb's potential as an enabling tool.

Linking Resources to People and Organisations

SEweb is designed, in the main, to feature Australian content. However, it also recognises that content on its own can be less than useful. What may be needed are technical and other skills to transform content into action and results. Additionally, perusal of content may lead to further questions, or demand discussion and argument, both of which require human contact.

Hence, SEweb stresses the link between content and people. All material is owned or managed by someone, and that person or organisation can be contacted if further information is required. SEweb's pages contain copious pointers to people and organisations to facilitate easy contact.

Intellectual Property Issues

It has become clear that there are some limitations to what material might be placed on the SEweb server (or, indeed, any public Web server) because of concerns with intellectual property (IP) rights.

The first problem relates to published papers. Much of the more detailed research material has been, or will be, published in the appropriate scholarly journals. This limits the author's ability to post full-text versions, and therefore summaries or abstracts have to be used.

The second major IP area relates to educational and tutorial material. With increased funding constraints on universities, many institutions look to short courses to provide additional funding. Also, competition for students has increased and the content of a course can be a significant factor in attracting the right students. Commercial organisations, of course, derive revenue from training and training materials. The net result of these pressures is that course content now has more visible value than it had before, and there can be a reluctance to make this available through a no-charge medium.

There is no easy answer to the above problems. However, the current emphasis on Australian content, clear local ownership of material, and inclusion of contacts and organisations, will ensure that any user of SEweb will be able to negotiate directly with the IP owner to ensure a mutually beneficial outcome.

Point-Of-View

As the Information Superhighway develops, one can expect to see increasing volumes of content. Not all of this will be of particular note nor of adequate quality. This will increase the difficulties associated with finding material and discriminating between useful content and noise. There will be technologies such as knowbots and intelligent agents to assist in the process of discovery and navigation.

However, this is not the whole issue, as Paul Saffo (Saf94) from the Institute for the Future notes:

It is not content but context that will matter most a decade or so from now. The scarce resource will not be stuff, but point of view.

SEweb anticipates this situation. A major role for sub-editors is context and point of view. They are the experts in their particular field, and the filtering, organisation and interpretation of material are what sets SEweb apart from simple collections of material and pointers.

Conclusion

Returning to the theme of shovels and holes, it is now necessary to attempt to answer the original question of whether the World Wide Web is a shovel or a hole

Firstly, SEweb (CSIRO95) has demonstrated that shovels are not much use without planning and hard work. Secondly, the project has indicated that the hole dug by the shovel is indeed useful, but only if filled with useful content, au.

SEweb will prove to be a useful and effective service. It will be so because of the attention paid to quality content, effective collaboration, and the addition of point-of-view.

References

[Fra95]
Frater R: Address to CeBIT `95, CSIRO, 1995.
[IEEE94]
IEEE: Standards Collection: Software Engineering, IEEE, 1994.
[Hor95]
Horan P: Personal communication, Deakin University, 1995.
[Blu92]
Blum B I: Software Engineering: a Holistic View, New York, Oxford University Press, 1992.
[RFC1206]
Malkin G & Marine A: RFC1206 Answers to Commonly asked "New Internet User" Questions, Network Working Group, 1991.
[RFC1207]
Malkin G, Marine A & Reynolds J: RFC1207 Answers to Commonly asked "Experienced Internet User" Questions, Network Working Group, 1991.
[RFC1462]
Krol E & Hoffman E: RFC 1462 FYI on "What is the Internet?", Network Working Group, 1993.
[Saf94]
Saffo P: It's the Context, Stupid, in WIRED Online, 1994.
[CSIRO95]
CSIRO: SEweb Home Page, <http://seweb.dit.csiro.au/>, 1995.

Acknowledgments

The author wishes to thank the Department of Industry, Science and Technology for sponsoring the SEweb pilot project, and CSIRO for providing a supportive environment for its development. Special thanks are due to the many academics within the university sector who have unstintingly provided content for SEweb, and the industry partners who have reviewed the service and provided their comments and opinions. Particular thanks are due to John O'Callaghan and Phil McCrea of CSIRO for their leadership during the project.

Author Information

Peter Outteridge is a director of Internet Information Services, a consulting company established to provide impartial business and technical advice on the Internet and its emerging technologies. He has over 25 years experience in IT working with mainframes, PCs and telecommunications networks in the public and private sectors. As a senior manger in the higher education sector for some six years, he has managed and used Internet services during a period of rapid growth. Currently his company is undertaking project management for the SEweb pilot project.