Show: Total No of Items:


WebZIP Tasks

Task Report:
Category:
Title:

Show: Items per page Total No of Items:
Status | Title | URL | File Type | Size | Last Modified | Download Result | Level | Referrer

© 2000 Spidersoft. All rights reserved.


Status,Title,URL,LocalFile,FileType,Size,LastModified,DownloadResult,Level,Referrer
"Parsed","Data Structures and Algorithms with Object-Oriented Design Patterns in C++","http://www.brpreiss.com/books/opus4/html/book.html","../www.brpreiss.com\books\opus4\html\book.html","HTML","4700"," Sun, 09 Dec 2001 00:02:32 GMT","200 OK","0",""
"Saved","","http://www.brpreiss.com/books/opus4/icons/cover75.gif","../www.brpreiss.com\books\opus4\icons\cover75.gif","Images","7696"," Sun, 09 Dec 2001 01:05:14 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Colophon","http://www.brpreiss.com/books/opus4/html/page1.html","../www.brpreiss.com\books\opus4\html\page1.html","HTML","2668"," Sun, 09 Dec 2001 00:02:35 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/next_motif.gif","../www.brpreiss.com\books\opus4\icons\next_motif.gif","Images","172"," Sun, 09 Dec 2001 01:05:35 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Data Structures and Algorithms with Object-Oriented Design Patterns in C++","http://www.brpreiss.com/books/opus4/main.html","../www.brpreiss.com\books\opus4\main.html","HTML","338"," Sat, 08 Dec 2001 23:48:25 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/up_motif.gif","../www.brpreiss.com\books\opus4\icons\up_motif.gif","Images","145"," Sun, 09 Dec 2001 01:05:45 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/previous_motif_gr.gif","../www.brpreiss.com\books\opus4\icons\previous_motif_gr.gif","Images","220"," Sun, 09 Dec 2001 01:05:41 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Contents","http://www.brpreiss.com/books/opus4/html/page9.html","../www.brpreiss.com\books\opus4\html\page9.html","HTML","69098"," Sun, 09 Dec 2001 00:04:04 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/contents_motif.gif","../www.brpreiss.com\books\opus4\icons\contents_motif.gif","Images","225"," Sun, 09 Dec 2001 01:05:24 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Index","http://www.brpreiss.com/books/opus4/html/page620.html","../www.brpreiss.com\books\opus4\html\page620.html","HTML","68662"," Sun, 09 Dec 2001 01:05:09 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/index_motif.gif","../www.brpreiss.com\books\opus4\icons\index_motif.gif","Images","180"," Sun, 09 Dec 2001 01:05:30 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Dedication","http://www.brpreiss.com/books/opus4/html/page2.html","../www.brpreiss.com\books\opus4\html\page2.html","HTML","1985"," Sun, 09 Dec 2001 00:02:45 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Preface","http://www.brpreiss.com/books/opus4/html/page3.html","../www.brpreiss.com\books\opus4\html\page3.html","HTML","3369"," Sun, 09 Dec 2001 00:02:54 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Introduction","http://www.brpreiss.com/books/opus4/html/page10.html","../www.brpreiss.com\books\opus4\html\page10.html","HTML","2411"," Sun, 09 Dec 2001 00:02:35 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Algorithm Analysis","http://www.brpreiss.com/books/opus4/html/page34.html","../www.brpreiss.com\books\opus4\html\page34.html","HTML","5285"," Sun, 09 Dec 2001 00:02:59 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Asymptotic Notation","http://www.brpreiss.com/books/opus4/html/page56.html","../www.brpreiss.com\books\opus4\html\page56.html","HTML","4695"," Sun, 09 Dec 2001 00:03:26 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Foundational Data Structures","http://www.brpreiss.com/books/opus4/html/page79.html","../www.brpreiss.com\books\opus4\html\page79.html","HTML","3452"," Sun, 09 Dec 2001 00:03:54 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Data Types and Abstraction","http://www.brpreiss.com/books/opus4/html/page107.html","../www.brpreiss.com\books\opus4\html\page107.html","HTML","2978"," Sun, 09 Dec 2001 00:07:17 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Stacks, Queues and Deques","http://www.brpreiss.com/books/opus4/html/page130.html","../www.brpreiss.com\books\opus4\html\page130.html","HTML","4073"," Sun, 09 Dec 2001 00:07:43 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Ordered Lists and Sorted Lists","http://www.brpreiss.com/books/opus4/html/page166.html","../www.brpreiss.com\books\opus4\html\page166.html","HTML","3895"," Sun, 09 Dec 2001 00:08:59 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Hashing, Hash Tables and Scatter Tables","http://www.brpreiss.com/books/opus4/html/page203.html","../www.brpreiss.com\books\opus4\html\page203.html","HTML","4234"," Sun, 09 Dec 2001 00:09:42 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Trees","http://www.brpreiss.com/books/opus4/html/page250.html","../www.brpreiss.com\books\opus4\html\page250.html","HTML","5201"," Sun, 09 Dec 2001 00:24:55 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Search Trees","http://www.brpreiss.com/books/opus4/html/page299.html","../www.brpreiss.com\books\opus4\html\page299.html","HTML","3721"," Sun, 09 Dec 2001 00:25:57 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Heaps and Priority Queues","http://www.brpreiss.com/books/opus4/html/page352.html","../www.brpreiss.com\books\opus4\html\page352.html","HTML","4947"," Sun, 09 Dec 2001 00:27:11 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Sets, Multisets and Partitions","http://www.brpreiss.com/books/opus4/html/page387.html","../www.brpreiss.com\books\opus4\html\page387.html","HTML","9057"," Sun, 09 Dec 2001 00:28:03 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Dynamic Storage Allocation: The Other Kind of Heap","http://www.brpreiss.com/books/opus4/html/page416.html","../www.brpreiss.com\books\opus4\html\page416.html","HTML","3717"," Sun, 09 Dec 2001 00:28:39 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Algorithmic Patterns and Problem Solvers","http://www.brpreiss.com/books/opus4/html/page440.html","../www.brpreiss.com\books\opus4\html\page440.html","HTML","3737"," Sun, 09 Dec 2001 00:29:10 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Sorting Algorithms and Sorters","http://www.brpreiss.com/books/opus4/html/page483.html","../www.brpreiss.com\books\opus4\html\page483.html","HTML","2986"," Sun, 09 Dec 2001 00:30:06 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Graphs  and Graph Algorithms","http://www.brpreiss.com/books/opus4/html/page523.html","../www.brpreiss.com\books\opus4\html\page523.html","HTML","5429"," Sun, 09 Dec 2001 00:30:57 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","C++ and Object-Oriented Programming","http://www.brpreiss.com/books/opus4/html/page586.html","../www.brpreiss.com\books\opus4\html\page586.html","HTML","2826"," Sun, 09 Dec 2001 00:32:21 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Class Hierarchy Diagrams","http://www.brpreiss.com/books/opus4/html/page617.html","../www.brpreiss.com\books\opus4\html\page617.html","HTML","2298"," Sun, 09 Dec 2001 01:05:01 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Character Codes","http://www.brpreiss.com/books/opus4/html/page618.html","../www.brpreiss.com\books\opus4\html\page618.html","HTML","11392"," Sun, 09 Dec 2001 01:05:02 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","References","http://www.brpreiss.com/books/opus4/html/page619.html","../www.brpreiss.com\books\opus4\html\page619.html","HTML","9623"," Sun, 09 Dec 2001 01:05:05 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/bruno.gif","../www.brpreiss.com\books\opus4\icons\bruno.gif","Images","410"," Sun, 09 Dec 2001 01:05:12 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Copyright Notice","http://www.brpreiss.com/books/opus4/copyright.html","../www.brpreiss.com\books\opus4\copyright.html","HTML","791"," Sat, 08 Dec 2001 23:48:27 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Parsed","Bruno R. Preiss-Signature Page","http://www.brpreiss.com/books/opus4/signature.html","../www.brpreiss.com\books\opus4\signature.html","HTML","1001"," Sun, 09 Dec 2001 15:58:14 GMT","200 OK","1","http://www.brpreiss.com/books/opus4/html/book.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/previous_motif.gif","../www.brpreiss.com\books\opus4\icons\previous_motif.gif","Images","220"," Sun, 09 Dec 2001 01:05:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page1.html"
"Parsed","Online Course Materials","http://www.brpreiss.com/books/opus4/html/page8.html","../www.brpreiss.com\books\opus4\html\page8.html","HTML","2258"," Sun, 09 Dec 2001 00:03:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Goals","http://www.brpreiss.com/books/opus4/html/page4.html","../www.brpreiss.com\books\opus4\html\page4.html","HTML","3518"," Sun, 09 Dec 2001 00:03:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Approach","http://www.brpreiss.com/books/opus4/html/page5.html","../www.brpreiss.com\books\opus4\html\page5.html","HTML","2926"," Sun, 09 Dec 2001 00:03:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Outline","http://www.brpreiss.com/books/opus4/html/page6.html","../www.brpreiss.com\books\opus4\html\page6.html","HTML","9763"," Sun, 09 Dec 2001 00:03:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Suggested Course Outline","http://www.brpreiss.com/books/opus4/html/page7.html","../www.brpreiss.com\books\opus4\html\page7.html","HTML","5033"," Sun, 09 Dec 2001 00:03:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","What This Book Is About","http://www.brpreiss.com/books/opus4/html/page11.html","../www.brpreiss.com\books\opus4\html\page11.html","HTML","2953"," Sun, 09 Dec 2001 00:02:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Object-Oriented Design","http://www.brpreiss.com/books/opus4/html/page12.html","../www.brpreiss.com\books\opus4\html\page12.html","HTML","3009"," Sun, 09 Dec 2001 00:02:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstraction","http://www.brpreiss.com/books/opus4/html/page13.html","../www.brpreiss.com\books\opus4\html\page13.html","HTML","2650"," Sun, 09 Dec 2001 00:02:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Encapsulation","http://www.brpreiss.com/books/opus4/html/page14.html","../www.brpreiss.com\books\opus4\html\page14.html","HTML","2822"," Sun, 09 Dec 2001 00:02:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Object Hierarchies and Design Patterns","http://www.brpreiss.com/books/opus4/html/page15.html","../www.brpreiss.com\books\opus4\html\page15.html","HTML","4172"," Sun, 09 Dec 2001 00:02:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Containers","http://www.brpreiss.com/books/opus4/html/page16.html","../www.brpreiss.com\books\opus4\html\page16.html","HTML","2087"," Sun, 09 Dec 2001 00:02:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Iterators","http://www.brpreiss.com/books/opus4/html/page17.html","../www.brpreiss.com\books\opus4\html\page17.html","HTML","2046"," Sun, 09 Dec 2001 00:02:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Visitors","http://www.brpreiss.com/books/opus4/html/page18.html","../www.brpreiss.com\books\opus4\html\page18.html","HTML","2127"," Sun, 09 Dec 2001 00:02:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Adapters","http://www.brpreiss.com/books/opus4/html/page19.html","../www.brpreiss.com\books\opus4\html\page19.html","HTML","2041"," Sun, 09 Dec 2001 00:02:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Singletons","http://www.brpreiss.com/books/opus4/html/page20.html","../www.brpreiss.com\books\opus4\html\page20.html","HTML","1975"," Sun, 09 Dec 2001 00:02:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Features of C++ You Need to Know","http://www.brpreiss.com/books/opus4/html/page21.html","../www.brpreiss.com\books\opus4\html\page21.html","HTML","2894"," Sun, 09 Dec 2001 00:02:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Variables","http://www.brpreiss.com/books/opus4/html/page22.html","../www.brpreiss.com\books\opus4\html\page22.html","HTML","2061"," Sun, 09 Dec 2001 00:02:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Parameter Passing","http://www.brpreiss.com/books/opus4/html/page23.html","../www.brpreiss.com\books\opus4\html\page23.html","HTML","2136"," Sun, 09 Dec 2001 00:02:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Pointers","http://www.brpreiss.com/books/opus4/html/page24.html","../www.brpreiss.com\books\opus4\html\page24.html","HTML","2185"," Sun, 09 Dec 2001 00:02:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Classes and Objects","http://www.brpreiss.com/books/opus4/html/page25.html","../www.brpreiss.com\books\opus4\html\page25.html","HTML","2313"," Sun, 09 Dec 2001 00:02:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inheritance","http://www.brpreiss.com/books/opus4/html/page26.html","../www.brpreiss.com\books\opus4\html\page26.html","HTML","2223"," Sun, 09 Dec 2001 00:02:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Other Features","http://www.brpreiss.com/books/opus4/html/page27.html","../www.brpreiss.com\books\opus4\html\page27.html","HTML","1993"," Sun, 09 Dec 2001 00:02:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","How This Book Is Organized","http://www.brpreiss.com/books/opus4/html/page28.html","../www.brpreiss.com\books\opus4\html\page28.html","HTML","2391"," Sun, 09 Dec 2001 00:02:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Models and Asymptotic Analysis","http://www.brpreiss.com/books/opus4/html/page29.html","../www.brpreiss.com\books\opus4\html\page29.html","HTML","2697"," Sun, 09 Dec 2001 00:02:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Foundational Data Structures","http://www.brpreiss.com/books/opus4/html/page30.html","../www.brpreiss.com\books\opus4\html\page30.html","HTML","2266"," Sun, 09 Dec 2001 00:02:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstract Data Types and the Class Hierarchy","http://www.brpreiss.com/books/opus4/html/page31.html","../www.brpreiss.com\books\opus4\html\page31.html","HTML","2396"," Sun, 09 Dec 2001 00:02:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Data Structures","http://www.brpreiss.com/books/opus4/html/page32.html","../www.brpreiss.com\books\opus4\html\page32.html","HTML","3770"," Sun, 09 Dec 2001 00:02:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Algorithms","http://www.brpreiss.com/books/opus4/html/page33.html","../www.brpreiss.com\books\opus4\html\page33.html","HTML","2899"," Sun, 09 Dec 2001 00:02:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Detailed Model of the Computer","http://www.brpreiss.com/books/opus4/html/page35.html","../www.brpreiss.com\books\opus4\html\page35.html","HTML","3989"," Sun, 09 Dec 2001 00:03:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Basic Axioms","http://www.brpreiss.com/books/opus4/html/page36.html","../www.brpreiss.com\books\opus4\html\page36.html","HTML","8150"," Sun, 09 Dec 2001 00:03:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Simple Example-Arithmetic Series Summation","http://www.brpreiss.com/books/opus4/html/page37.html","../www.brpreiss.com\books\opus4\html\page37.html","HTML","7404"," Sun, 09 Dec 2001 00:03:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Subscripting Operations","http://www.brpreiss.com/books/opus4/html/page38.html","../www.brpreiss.com\books\opus4\html\page38.html","HTML","3132"," Sun, 09 Dec 2001 00:03:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Another Example-Horner's Rule","http://www.brpreiss.com/books/opus4/html/page39.html","../www.brpreiss.com\books\opus4\html\page39.html","HTML","6479"," Sun, 09 Dec 2001 00:03:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Analyzing Recursive Functions","http://www.brpreiss.com/books/opus4/html/page40.html","../www.brpreiss.com\books\opus4\html\page40.html","HTML","6750"," Sun, 09 Dec 2001 00:03:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Solving Recurrence Relations-Repeated Substitution","http://www.brpreiss.com/books/opus4/html/page41.html","../www.brpreiss.com\books\opus4\html\page41.html","HTML","5238"," Sun, 09 Dec 2001 00:03:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Yet Another Example-Finding the Largest Element of an Array","http://www.brpreiss.com/books/opus4/html/page42.html","../www.brpreiss.com\books\opus4\html\page42.html","HTML","7450"," Sun, 09 Dec 2001 00:03:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Running Times","http://www.brpreiss.com/books/opus4/html/page43.html","../www.brpreiss.com\books\opus4\html\page43.html","HTML","6255"," Sun, 09 Dec 2001 00:03:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Harmonic Numbers","http://www.brpreiss.com/books/opus4/html/page44.html","../www.brpreiss.com\books\opus4\html\page44.html","HTML","8190"," Sun, 09 Dec 2001 00:03:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Best-Case and Worst-Case Running Times","http://www.brpreiss.com/books/opus4/html/page45.html","../www.brpreiss.com\books\opus4\html\page45.html","HTML","3841"," Sun, 09 Dec 2001 00:03:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Last Axiom","http://www.brpreiss.com/books/opus4/html/page46.html","../www.brpreiss.com\books\opus4\html\page46.html","HTML","3247"," Sun, 09 Dec 2001 00:03:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Simplified Model of the Computer","http://www.brpreiss.com/books/opus4/html/page47.html","../www.brpreiss.com\books\opus4\html\page47.html","HTML","5660"," Sun, 09 Dec 2001 00:03:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","An Example-Geometric Series Summation","http://www.brpreiss.com/books/opus4/html/page48.html","../www.brpreiss.com\books\opus4\html\page48.html","HTML","6227"," Sun, 09 Dec 2001 00:03:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Arithmetic Series Summation","http://www.brpreiss.com/books/opus4/html/page49.html","../www.brpreiss.com\books\opus4\html\page49.html","HTML","3808"," Sun, 09 Dec 2001 00:03:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Geometric Series Summation Again","http://www.brpreiss.com/books/opus4/html/page50.html","../www.brpreiss.com\books\opus4\html\page50.html","HTML","6216"," Sun, 09 Dec 2001 00:03:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Geometric Series Summation","http://www.brpreiss.com/books/opus4/html/page51.html","../www.brpreiss.com\books\opus4\html\page51.html","HTML","2818"," Sun, 09 Dec 2001 00:03:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Computing Powers","http://www.brpreiss.com/books/opus4/html/page52.html","../www.brpreiss.com\books\opus4\html\page52.html","HTML","15696"," Sun, 09 Dec 2001 00:03:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Geometric Series Summation Yet Again","http://www.brpreiss.com/books/opus4/html/page53.html","../www.brpreiss.com\books\opus4\html\page53.html","HTML","7801"," Sun, 09 Dec 2001 00:03:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page54.html","../www.brpreiss.com\books\opus4\html\page54.html","HTML","5432"," Sun, 09 Dec 2001 00:03:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page55.html","../www.brpreiss.com\books\opus4\html\page55.html","HTML","6443"," Sun, 09 Dec 2001 00:03:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","An Asymptotic Upper Bound-Big Oh","http://www.brpreiss.com/books/opus4/html/page57.html","../www.brpreiss.com\books\opus4\html\page57.html","HTML","3692"," Sun, 09 Dec 2001 00:03:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Simple Example","http://www.brpreiss.com/books/opus4/html/page58.html","../www.brpreiss.com\books\opus4\html\page58.html","HTML","4923"," Sun, 09 Dec 2001 00:03:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Big Oh Fallacies and Pitfalls","http://www.brpreiss.com/books/opus4/html/page59.html","../www.brpreiss.com\books\opus4\html\page59.html","HTML","4629"," Sun, 09 Dec 2001 00:03:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Properties of Big Oh","http://www.brpreiss.com/books/opus4/html/page60.html","../www.brpreiss.com\books\opus4\html\page60.html","HTML","22124"," Sun, 09 Dec 2001 00:03:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Polynomials","http://www.brpreiss.com/books/opus4/html/page61.html","../www.brpreiss.com\books\opus4\html\page61.html","HTML","4980"," Sun, 09 Dec 2001 00:03:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Logarithms","http://www.brpreiss.com/books/opus4/html/page62.html","../www.brpreiss.com\books\opus4\html\page62.html","HTML","4992"," Sun, 09 Dec 2001 00:03:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tight Big Oh Bounds","http://www.brpreiss.com/books/opus4/html/page63.html","../www.brpreiss.com\books\opus4\html\page63.html","HTML","5174"," Sun, 09 Dec 2001 00:03:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","More Big Oh Fallacies and Pitfalls","http://www.brpreiss.com/books/opus4/html/page64.html","../www.brpreiss.com\books\opus4\html\page64.html","HTML","11852"," Sun, 09 Dec 2001 00:03:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Conventions for Writing Big Oh Expressions","http://www.brpreiss.com/books/opus4/html/page65.html","../www.brpreiss.com\books\opus4\html\page65.html","HTML","5584"," Sun, 09 Dec 2001 00:03:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","An Asymptotic Lower Bound-Omega","http://www.brpreiss.com/books/opus4/html/page66.html","../www.brpreiss.com\books\opus4\html\page66.html","HTML","3562"," Sun, 09 Dec 2001 00:03:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Simple Example","http://www.brpreiss.com/books/opus4/html/page67.html","../www.brpreiss.com\books\opus4\html\page67.html","HTML","4773"," Sun, 09 Dec 2001 00:03:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","About Polynomials Again","http://www.brpreiss.com/books/opus4/html/page68.html","../www.brpreiss.com\books\opus4\html\page68.html","HTML","6562"," Sun, 09 Dec 2001 00:03:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","More Notation-Theta and Little Oh","http://www.brpreiss.com/books/opus4/html/page69.html","../www.brpreiss.com\books\opus4\html\page69.html","HTML","6267"," Sun, 09 Dec 2001 00:03:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Asymptotic Analysis of Algorithms","http://www.brpreiss.com/books/opus4/html/page70.html","../www.brpreiss.com\books\opus4\html\page70.html","HTML","10321"," Sun, 09 Dec 2001 00:03:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Rules For Big Oh Analysis of Running Time","http://www.brpreiss.com/books/opus4/html/page71.html","../www.brpreiss.com\books\opus4\html\page71.html","HTML","6245"," Sun, 09 Dec 2001 00:03:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Prefix Sums","http://www.brpreiss.com/books/opus4/html/page72.html","../www.brpreiss.com\books\opus4\html\page72.html","HTML","8168"," Sun, 09 Dec 2001 00:03:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Fibonacci Numbers","http://www.brpreiss.com/books/opus4/html/page73.html","../www.brpreiss.com\books\opus4\html\page73.html","HTML","16832"," Sun, 09 Dec 2001 00:03:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Bucket Sort","http://www.brpreiss.com/books/opus4/html/page74.html","../www.brpreiss.com\books\opus4\html\page74.html","HTML","8053"," Sun, 09 Dec 2001 00:03:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Reality Check","http://www.brpreiss.com/books/opus4/html/page75.html","../www.brpreiss.com\books\opus4\html\page75.html","HTML","9416"," Sun, 09 Dec 2001 00:03:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Checking Your Analysis","http://www.brpreiss.com/books/opus4/html/page76.html","../www.brpreiss.com\books\opus4\html\page76.html","HTML","4172"," Sun, 09 Dec 2001 00:03:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page77.html","../www.brpreiss.com\books\opus4\html\page77.html","HTML","11592"," Sun, 09 Dec 2001 00:03:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page78.html","../www.brpreiss.com\books\opus4\html\page78.html","HTML","3462"," Sun, 09 Dec 2001 00:03:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Dynamic Arrays","http://www.brpreiss.com/books/opus4/html/page80.html","../www.brpreiss.com\books\opus4\html\page80.html","HTML","7117"," Sun, 09 Dec 2001 00:03:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Default Constructor","http://www.brpreiss.com/books/opus4/html/page81.html","../www.brpreiss.com\books\opus4\html\page81.html","HTML","2665"," Sun, 09 Dec 2001 00:03:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Constructor","http://www.brpreiss.com/books/opus4/html/page82.html","../www.brpreiss.com\books\opus4\html\page82.html","HTML","4172"," Sun, 09 Dec 2001 00:03:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Copy Constructor","http://www.brpreiss.com/books/opus4/html/page83.html","../www.brpreiss.com\books\opus4\html\page83.html","HTML","5110"," Sun, 09 Dec 2001 00:03:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Destructor","http://www.brpreiss.com/books/opus4/html/page84.html","../www.brpreiss.com\books\opus4\html\page84.html","HTML","3124"," Sun, 09 Dec 2001 00:03:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Member Functions","http://www.brpreiss.com/books/opus4/html/page85.html","../www.brpreiss.com\books\opus4\html\page85.html","HTML","2903"," Sun, 09 Dec 2001 00:04:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Subscripting Operator","http://www.brpreiss.com/books/opus4/html/page86.html","../www.brpreiss.com\books\opus4\html\page86.html","HTML","3464"," Sun, 09 Dec 2001 00:08:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Resizing an Array","http://www.brpreiss.com/books/opus4/html/page87.html","../www.brpreiss.com\books\opus4\html\page87.html","HTML","3390"," Sun, 09 Dec 2001 00:08:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Singly-Linked Lists","http://www.brpreiss.com/books/opus4/html/page88.html","../www.brpreiss.com\books\opus4\html\page88.html","HTML","9758"," Sun, 09 Dec 2001 00:08:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","An Implementation","http://www.brpreiss.com/books/opus4/html/page89.html","../www.brpreiss.com\books\opus4\html\page89.html","HTML","4624"," Sun, 09 Dec 2001 00:08:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","List Elements","http://www.brpreiss.com/books/opus4/html/page90.html","../www.brpreiss.com\books\opus4\html\page90.html","HTML","3219"," Sun, 09 Dec 2001 00:08:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Default Constructor","http://www.brpreiss.com/books/opus4/html/page91.html","../www.brpreiss.com\books\opus4\html\page91.html","HTML","2658"," Sun, 09 Dec 2001 00:08:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Destructor and Purge Member Function","http://www.brpreiss.com/books/opus4/html/page92.html","../www.brpreiss.com\books\opus4\html\page92.html","HTML","3449"," Sun, 09 Dec 2001 00:08:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Accessors","http://www.brpreiss.com/books/opus4/html/page93.html","../www.brpreiss.com\books\opus4\html\page93.html","HTML","2562"," Sun, 09 Dec 2001 00:08:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","First and Last Functions","http://www.brpreiss.com/books/opus4/html/page94.html","../www.brpreiss.com\books\opus4\html\page94.html","HTML","3239"," Sun, 09 Dec 2001 00:08:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Prepend","http://www.brpreiss.com/books/opus4/html/page95.html","../www.brpreiss.com\books\opus4\html\page95.html","HTML","3781"," Sun, 09 Dec 2001 00:08:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Append","http://www.brpreiss.com/books/opus4/html/page96.html","../www.brpreiss.com\books\opus4\html\page96.html","HTML","3146"," Sun, 09 Dec 2001 00:08:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Copy Constructor and Assignment Operator","http://www.brpreiss.com/books/opus4/html/page97.html","../www.brpreiss.com\books\opus4\html\page97.html","HTML","3807"," Sun, 09 Dec 2001 00:08:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Extract","http://www.brpreiss.com/books/opus4/html/page98.html","../www.brpreiss.com\books\opus4\html\page98.html","HTML","5610"," Sun, 09 Dec 2001 00:08:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","InsertAfter and InsertBefore","http://www.brpreiss.com/books/opus4/html/page99.html","../www.brpreiss.com\books\opus4\html\page99.html","HTML","3556"," Sun, 09 Dec 2001 00:08:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Multi-Dimensional Arrays","http://www.brpreiss.com/books/opus4/html/page100.html","../www.brpreiss.com\books\opus4\html\page100.html","HTML","3869"," Sun, 09 Dec 2001 00:07:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Subscript Calculations","http://www.brpreiss.com/books/opus4/html/page101.html","../www.brpreiss.com\books\opus4\html\page101.html","HTML","6628"," Sun, 09 Dec 2001 00:07:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Two-Dimensional Array Implementation","http://www.brpreiss.com/books/opus4/html/page102.html","../www.brpreiss.com\books\opus4\html\page102.html","HTML","4572"," Sun, 09 Dec 2001 00:07:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Multi-Dimensional Subscripting in C++","http://www.brpreiss.com/books/opus4/html/page103.html","../www.brpreiss.com\books\opus4\html\page103.html","HTML","4776"," Sun, 09 Dec 2001 00:07:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Canonical Matrix Multiplication","http://www.brpreiss.com/books/opus4/html/page104.html","../www.brpreiss.com\books\opus4\html\page104.html","HTML","9098"," Sun, 09 Dec 2001 00:07:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page105.html","../www.brpreiss.com\books\opus4\html\page105.html","HTML","6182"," Sun, 09 Dec 2001 00:07:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page106.html","../www.brpreiss.com\books\opus4\html\page106.html","HTML","4521"," Sun, 09 Dec 2001 00:07:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstract Data Types","http://www.brpreiss.com/books/opus4/html/page108.html","../www.brpreiss.com\books\opus4\html\page108.html","HTML","6583"," Sun, 09 Dec 2001 00:07:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Design Patterns","http://www.brpreiss.com/books/opus4/html/page109.html","../www.brpreiss.com\books\opus4\html\page109.html","HTML","3833"," Sun, 09 Dec 2001 00:07:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Class Hierarchy","http://www.brpreiss.com/books/opus4/html/page110.html","../www.brpreiss.com\books\opus4\html\page110.html","HTML","5114"," Sun, 09 Dec 2001 00:07:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Objects","http://www.brpreiss.com/books/opus4/html/page111.html","../www.brpreiss.com\books\opus4\html\page111.html","HTML","7856"," Sun, 09 Dec 2001 00:07:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page112.html","../www.brpreiss.com\books\opus4\html\page112.html","HTML","3291"," Sun, 09 Dec 2001 00:07:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The NullObject Singleton Class","http://www.brpreiss.com/books/opus4/html/page113.html","../www.brpreiss.com\books\opus4\html\page113.html","HTML","3000"," Sun, 09 Dec 2001 00:07:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page114.html","../www.brpreiss.com\books\opus4\html\page114.html","HTML","2809"," Sun, 09 Dec 2001 00:07:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Object Wrappers for the Built-In Types","http://www.brpreiss.com/books/opus4/html/page115.html","../www.brpreiss.com\books\opus4\html\page115.html","HTML","3455"," Sun, 09 Dec 2001 00:07:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page116.html","../www.brpreiss.com\books\opus4\html\page116.html","HTML","5804"," Sun, 09 Dec 2001 00:07:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Containers","http://www.brpreiss.com/books/opus4/html/page117.html","../www.brpreiss.com\books\opus4\html\page117.html","HTML","6893"," Sun, 09 Dec 2001 00:07:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Visitors","http://www.brpreiss.com/books/opus4/html/page118.html","../www.brpreiss.com\books\opus4\html\page118.html","HTML","5314"," Sun, 09 Dec 2001 00:07:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The IsDone Member Function","http://www.brpreiss.com/books/opus4/html/page119.html","../www.brpreiss.com\books\opus4\html\page119.html","HTML","5107"," Sun, 09 Dec 2001 00:07:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Container Class Default Put Member Function","http://www.brpreiss.com/books/opus4/html/page120.html","../www.brpreiss.com\books\opus4\html\page120.html","HTML","3318"," Sun, 09 Dec 2001 00:07:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Iterators","http://www.brpreiss.com/books/opus4/html/page121.html","../www.brpreiss.com\books\opus4\html\page121.html","HTML","7466"," Sun, 09 Dec 2001 00:07:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The NullIterator Class","http://www.brpreiss.com/books/opus4/html/page122.html","../www.brpreiss.com\books\opus4\html\page122.html","HTML","4100"," Sun, 09 Dec 2001 00:07:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Direct vs. Indirect Containment","http://www.brpreiss.com/books/opus4/html/page123.html","../www.brpreiss.com\books\opus4\html\page123.html","HTML","4331"," Sun, 09 Dec 2001 00:07:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Ownership of Contained Objects","http://www.brpreiss.com/books/opus4/html/page124.html","../www.brpreiss.com\books\opus4\html\page124.html","HTML","6910"," Sun, 09 Dec 2001 00:07:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Associations","http://www.brpreiss.com/books/opus4/html/page125.html","../www.brpreiss.com\books\opus4\html\page125.html","HTML","3855"," Sun, 09 Dec 2001 00:07:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page126.html","../www.brpreiss.com\books\opus4\html\page126.html","HTML","4267"," Sun, 09 Dec 2001 00:07:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Searchable Containers","http://www.brpreiss.com/books/opus4/html/page127.html","../www.brpreiss.com\books\opus4\html\page127.html","HTML","5269"," Sun, 09 Dec 2001 00:07:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page128.html","../www.brpreiss.com\books\opus4\html\page128.html","HTML","5546"," Sun, 09 Dec 2001 00:07:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page129.html","../www.brpreiss.com\books\opus4\html\page129.html","HTML","5046"," Sun, 09 Dec 2001 00:07:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Stacks","http://www.brpreiss.com/books/opus4/html/page131.html","../www.brpreiss.com\books\opus4\html\page131.html","HTML","4964"," Sun, 09 Dec 2001 00:07:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page132.html","../www.brpreiss.com\books\opus4\html\page132.html","HTML","3264"," Sun, 09 Dec 2001 00:07:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page133.html","../www.brpreiss.com\books\opus4\html\page133.html","HTML","2286"," Sun, 09 Dec 2001 00:07:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page134.html","../www.brpreiss.com\books\opus4\html\page134.html","HTML","3602"," Sun, 09 Dec 2001 00:07:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Push, Pop, and Top Member Functions","http://www.brpreiss.com/books/opus4/html/page135.html","../www.brpreiss.com\books\opus4\html\page135.html","HTML","4452"," Sun, 09 Dec 2001 00:07:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Accept Member Function","http://www.brpreiss.com/books/opus4/html/page136.html","../www.brpreiss.com\books\opus4\html\page136.html","HTML","3653"," Sun, 09 Dec 2001 00:07:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Iterator","http://www.brpreiss.com/books/opus4/html/page137.html","../www.brpreiss.com\books\opus4\html\page137.html","HTML","5537"," Sun, 09 Dec 2001 00:07:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page138.html","../www.brpreiss.com\books\opus4\html\page138.html","HTML","3497"," Sun, 09 Dec 2001 00:07:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page139.html","../www.brpreiss.com\books\opus4\html\page139.html","HTML","2404"," Sun, 09 Dec 2001 00:07:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page140.html","../www.brpreiss.com\books\opus4\html\page140.html","HTML","3609"," Sun, 09 Dec 2001 00:07:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Push, Pop, and Top Member Functions","http://www.brpreiss.com/books/opus4/html/page141.html","../www.brpreiss.com\books\opus4\html\page141.html","HTML","5066"," Sun, 09 Dec 2001 00:07:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Accept Member Function","http://www.brpreiss.com/books/opus4/html/page142.html","../www.brpreiss.com\books\opus4\html\page142.html","HTML","3509"," Sun, 09 Dec 2001 00:07:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Iterator","http://www.brpreiss.com/books/opus4/html/page143.html","../www.brpreiss.com\books\opus4\html\page143.html","HTML","4782"," Sun, 09 Dec 2001 00:07:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page144.html","../www.brpreiss.com\books\opus4\html\page144.html","HTML","6416"," Sun, 09 Dec 2001 00:07:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Evaluating Postfix Expressions","http://www.brpreiss.com/books/opus4/html/page145.html","../www.brpreiss.com\books\opus4\html\page145.html","HTML","3268"," Sun, 09 Dec 2001 00:07:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page146.html","../www.brpreiss.com\books\opus4\html\page146.html","HTML","3916"," Sun, 09 Dec 2001 00:08:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Queues","http://www.brpreiss.com/books/opus4/html/page147.html","../www.brpreiss.com\books\opus4\html\page147.html","HTML","4131"," Sun, 09 Dec 2001 00:08:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page148.html","../www.brpreiss.com\books\opus4\html\page148.html","HTML","2850"," Sun, 09 Dec 2001 00:08:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page149.html","../www.brpreiss.com\books\opus4\html\page149.html","HTML","5180"," Sun, 09 Dec 2001 00:08:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page150.html","../www.brpreiss.com\books\opus4\html\page150.html","HTML","3660"," Sun, 09 Dec 2001 00:08:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Head, Enqueue, and Dequeue Member Functions","http://www.brpreiss.com/books/opus4/html/page151.html","../www.brpreiss.com\books\opus4\html\page151.html","HTML","4084"," Sun, 09 Dec 2001 00:08:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page152.html","../www.brpreiss.com\books\opus4\html\page152.html","HTML","2882"," Sun, 09 Dec 2001 00:08:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page153.html","../www.brpreiss.com\books\opus4\html\page153.html","HTML","2117"," Sun, 09 Dec 2001 00:08:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page154.html","../www.brpreiss.com\books\opus4\html\page154.html","HTML","3061"," Sun, 09 Dec 2001 00:08:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Head, Enqueue and Dequeue Member Functions","http://www.brpreiss.com/books/opus4/html/page155.html","../www.brpreiss.com\books\opus4\html\page155.html","HTML","3979"," Sun, 09 Dec 2001 00:08:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page156.html","../www.brpreiss.com\books\opus4\html\page156.html","HTML","6912"," Sun, 09 Dec 2001 00:08:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page157.html","../www.brpreiss.com\books\opus4\html\page157.html","HTML","2994"," Sun, 09 Dec 2001 00:08:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Deques","http://www.brpreiss.com/books/opus4/html/page158.html","../www.brpreiss.com\books\opus4\html\page158.html","HTML","6930"," Sun, 09 Dec 2001 00:08:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page159.html","../www.brpreiss.com\books\opus4\html\page159.html","HTML","3424"," Sun, 09 Dec 2001 00:08:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tail, EnqueueHead,    and DequeueTail Member Functions","http://www.brpreiss.com/books/opus4/html/page160.html","../www.brpreiss.com\books\opus4\html\page160.html","HTML","4143"," Sun, 09 Dec 2001 00:08:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page161.html","../www.brpreiss.com\books\opus4\html\page161.html","HTML","3482"," Sun, 09 Dec 2001 00:08:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tail, EnqueueHead,    and DequeueTail Member Functions","http://www.brpreiss.com/books/opus4/html/page162.html","../www.brpreiss.com\books\opus4\html\page162.html","HTML","4123"," Sun, 09 Dec 2001 00:08:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Doubly-Linked and Circular Lists","http://www.brpreiss.com/books/opus4/html/page163.html","../www.brpreiss.com\books\opus4\html\page163.html","HTML","6410"," Sun, 09 Dec 2001 00:08:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page164.html","../www.brpreiss.com\books\opus4\html\page164.html","HTML","7172"," Sun, 09 Dec 2001 00:08:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page165.html","../www.brpreiss.com\books\opus4\html\page165.html","HTML","9263"," Sun, 09 Dec 2001 00:08:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Ordered Lists","http://www.brpreiss.com/books/opus4/html/page167.html","../www.brpreiss.com\books\opus4\html\page167.html","HTML","6547"," Sun, 09 Dec 2001 00:09:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page168.html","../www.brpreiss.com\books\opus4\html\page168.html","HTML","3715"," Sun, 09 Dec 2001 00:09:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page169.html","../www.brpreiss.com\books\opus4\html\page169.html","HTML","2874"," Sun, 09 Dec 2001 00:09:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting and Accessing Items in a List","http://www.brpreiss.com/books/opus4/html/page170.html","../www.brpreiss.com\books\opus4\html\page170.html","HTML","4265"," Sun, 09 Dec 2001 00:09:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding Items in a List","http://www.brpreiss.com/books/opus4/html/page171.html","../www.brpreiss.com\books\opus4\html\page171.html","HTML","5364"," Sun, 09 Dec 2001 00:09:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a List","http://www.brpreiss.com/books/opus4/html/page172.html","../www.brpreiss.com\books\opus4\html\page172.html","HTML","4730"," Sun, 09 Dec 2001 00:09:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Positions of Items in a List","http://www.brpreiss.com/books/opus4/html/page173.html","../www.brpreiss.com\books\opus4\html\page173.html","HTML","3305"," Sun, 09 Dec 2001 00:09:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding the Position of an Item and Accessing by Position","http://www.brpreiss.com/books/opus4/html/page174.html","../www.brpreiss.com\books\opus4\html\page174.html","HTML","4756"," Sun, 09 Dec 2001 00:09:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting an Item at an Arbitrary Position","http://www.brpreiss.com/books/opus4/html/page175.html","../www.brpreiss.com\books\opus4\html\page175.html","HTML","5199"," Sun, 09 Dec 2001 00:09:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Arbitrary Items by Position","http://www.brpreiss.com/books/opus4/html/page176.html","../www.brpreiss.com\books\opus4\html\page176.html","HTML","3936"," Sun, 09 Dec 2001 00:09:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page177.html","../www.brpreiss.com\books\opus4\html\page177.html","HTML","3698"," Sun, 09 Dec 2001 00:09:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page178.html","../www.brpreiss.com\books\opus4\html\page178.html","HTML","2248"," Sun, 09 Dec 2001 00:09:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting and Accessing Items in a List","http://www.brpreiss.com/books/opus4/html/page179.html","../www.brpreiss.com\books\opus4\html\page179.html","HTML","4641"," Sun, 09 Dec 2001 00:09:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding Items in a List","http://www.brpreiss.com/books/opus4/html/page180.html","../www.brpreiss.com\books\opus4\html\page180.html","HTML","3648"," Sun, 09 Dec 2001 00:09:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a List","http://www.brpreiss.com/books/opus4/html/page181.html","../www.brpreiss.com\books\opus4\html\page181.html","HTML","2987"," Sun, 09 Dec 2001 00:09:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Positions of Items in a List","http://www.brpreiss.com/books/opus4/html/page182.html","../www.brpreiss.com\books\opus4\html\page182.html","HTML","3241"," Sun, 09 Dec 2001 00:09:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding the Position of an Item and Accessing by Position","http://www.brpreiss.com/books/opus4/html/page183.html","../www.brpreiss.com\books\opus4\html\page183.html","HTML","4166"," Sun, 09 Dec 2001 00:09:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting an Item at an Arbitrary Position","http://www.brpreiss.com/books/opus4/html/page184.html","../www.brpreiss.com\books\opus4\html\page184.html","HTML","3507"," Sun, 09 Dec 2001 00:09:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Arbitrary Items by Position","http://www.brpreiss.com/books/opus4/html/page185.html","../www.brpreiss.com\books\opus4\html\page185.html","HTML","3424"," Sun, 09 Dec 2001 00:09:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Performance Comparison:ListAsArray vs. ListAsLinkedList","http://www.brpreiss.com/books/opus4/html/page186.html","../www.brpreiss.com\books\opus4\html\page186.html","HTML","9298"," Sun, 09 Dec 2001 00:09:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page187.html","../www.brpreiss.com\books\opus4\html\page187.html","HTML","11492"," Sun, 09 Dec 2001 00:09:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Sorted Lists","http://www.brpreiss.com/books/opus4/html/page188.html","../www.brpreiss.com\books\opus4\html\page188.html","HTML","5080"," Sun, 09 Dec 2001 00:09:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page189.html","../www.brpreiss.com\books\opus4\html\page189.html","HTML","3514"," Sun, 09 Dec 2001 00:09:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items in a Sorted List","http://www.brpreiss.com/books/opus4/html/page190.html","../www.brpreiss.com\books\opus4\html\page190.html","HTML","4304"," Sun, 09 Dec 2001 00:09:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Locating Items in an Array-Binary Search","http://www.brpreiss.com/books/opus4/html/page191.html","../www.brpreiss.com\books\opus4\html\page191.html","HTML","4599"," Sun, 09 Dec 2001 00:09:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding Items in a Sorted List","http://www.brpreiss.com/books/opus4/html/page192.html","../www.brpreiss.com\books\opus4\html\page192.html","HTML","3853"," Sun, 09 Dec 2001 00:09:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a List","http://www.brpreiss.com/books/opus4/html/page193.html","../www.brpreiss.com\books\opus4\html\page193.html","HTML","3402"," Sun, 09 Dec 2001 00:09:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page194.html","../www.brpreiss.com\books\opus4\html\page194.html","HTML","3443"," Sun, 09 Dec 2001 00:09:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items in a Sorted List","http://www.brpreiss.com/books/opus4/html/page195.html","../www.brpreiss.com\books\opus4\html\page195.html","HTML","3973"," Sun, 09 Dec 2001 00:09:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Other Operations on Sorted Lists","http://www.brpreiss.com/books/opus4/html/page196.html","../www.brpreiss.com\books\opus4\html\page196.html","HTML","2183"," Sun, 09 Dec 2001 00:09:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Performance Comparison:SortedListAsArray vs. SortedListAsList","http://www.brpreiss.com/books/opus4/html/page197.html","../www.brpreiss.com\books\opus4\html\page197.html","HTML","5246"," Sun, 09 Dec 2001 00:09:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page198.html","../www.brpreiss.com\books\opus4\html\page198.html","HTML","3861"," Sun, 09 Dec 2001 00:09:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page199.html","../www.brpreiss.com\books\opus4\html\page199.html","HTML","4886"," Sun, 09 Dec 2001 00:09:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Analysis","http://www.brpreiss.com/books/opus4/html/page200.html","../www.brpreiss.com\books\opus4\html\page200.html","HTML","5443"," Sun, 09 Dec 2001 00:09:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page201.html","../www.brpreiss.com\books\opus4\html\page201.html","HTML","5873"," Sun, 09 Dec 2001 00:09:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page202.html","../www.brpreiss.com\books\opus4\html\page202.html","HTML","4228"," Sun, 09 Dec 2001 00:09:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hashing-The Basic Idea","http://www.brpreiss.com/books/opus4/html/page204.html","../www.brpreiss.com\books\opus4\html\page204.html","HTML","4730"," Sun, 09 Dec 2001 00:09:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example","http://www.brpreiss.com/books/opus4/html/page205.html","../www.brpreiss.com\books\opus4\html\page205.html","HTML","4904"," Sun, 09 Dec 2001 00:09:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Keys and Hash Functions","http://www.brpreiss.com/books/opus4/html/page206.html","../www.brpreiss.com\books\opus4\html\page206.html","HTML","4644"," Sun, 09 Dec 2001 00:09:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Avoiding Collisions","http://www.brpreiss.com/books/opus4/html/page207.html","../www.brpreiss.com\books\opus4\html\page207.html","HTML","2730"," Sun, 09 Dec 2001 00:09:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Spreading Keys Evenly","http://www.brpreiss.com/books/opus4/html/page208.html","../www.brpreiss.com\books\opus4\html\page208.html","HTML","3238"," Sun, 09 Dec 2001 00:09:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Ease of Computation","http://www.brpreiss.com/books/opus4/html/page209.html","../www.brpreiss.com\books\opus4\html\page209.html","HTML","2079"," Sun, 09 Dec 2001 00:09:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hashing Methods","http://www.brpreiss.com/books/opus4/html/page210.html","../www.brpreiss.com\books\opus4\html\page210.html","HTML","2574"," Sun, 09 Dec 2001 00:09:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Division Method","http://www.brpreiss.com/books/opus4/html/page211.html","../www.brpreiss.com\books\opus4\html\page211.html","HTML","4949"," Sun, 09 Dec 2001 00:09:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Middle Square Method","http://www.brpreiss.com/books/opus4/html/page212.html","../www.brpreiss.com\books\opus4\html\page212.html","HTML","5572"," Sun, 09 Dec 2001 00:09:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Multiplication Method","http://www.brpreiss.com/books/opus4/html/page213.html","../www.brpreiss.com\books\opus4\html\page213.html","HTML","4721"," Sun, 09 Dec 2001 00:09:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Fibonacci Hashing","http://www.brpreiss.com/books/opus4/html/page214.html","../www.brpreiss.com\books\opus4\html\page214.html","HTML","6086"," Sun, 09 Dec 2001 00:09:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hash Function Implementations","http://www.brpreiss.com/books/opus4/html/page215.html","../www.brpreiss.com\books\opus4\html\page215.html","HTML","5532"," Sun, 09 Dec 2001 00:09:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Integral Keys","http://www.brpreiss.com/books/opus4/html/page216.html","../www.brpreiss.com\books\opus4\html\page216.html","HTML","3036"," Sun, 09 Dec 2001 00:09:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Floating-Point Keys","http://www.brpreiss.com/books/opus4/html/page217.html","../www.brpreiss.com\books\opus4\html\page217.html","HTML","5882"," Sun, 09 Dec 2001 00:09:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Character String Keys","http://www.brpreiss.com/books/opus4/html/page218.html","../www.brpreiss.com\books\opus4\html\page218.html","HTML","18243"," Sun, 09 Dec 2001 00:10:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hashing Objects","http://www.brpreiss.com/books/opus4/html/page219.html","../www.brpreiss.com\books\opus4\html\page219.html","HTML","4704"," Sun, 09 Dec 2001 00:10:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hashing Containers","http://www.brpreiss.com/books/opus4/html/page220.html","../www.brpreiss.com\books\opus4\html\page220.html","HTML","4704"," Sun, 09 Dec 2001 00:10:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Using Associations","http://www.brpreiss.com/books/opus4/html/page221.html","../www.brpreiss.com\books\opus4\html\page221.html","HTML","4316"," Sun, 09 Dec 2001 00:10:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Hash Tables","http://www.brpreiss.com/books/opus4/html/page222.html","../www.brpreiss.com\books\opus4\html\page222.html","HTML","4469"," Sun, 09 Dec 2001 00:10:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Separate Chaining","http://www.brpreiss.com/books/opus4/html/page223.html","../www.brpreiss.com\books\opus4\html\page223.html","HTML","3738"," Sun, 09 Dec 2001 00:10:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page224.html","../www.brpreiss.com\books\opus4\html\page224.html","HTML","2440"," Sun, 09 Dec 2001 00:10:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page225.html","../www.brpreiss.com\books\opus4\html\page225.html","HTML","4438"," Sun, 09 Dec 2001 00:10:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting and Removing Items","http://www.brpreiss.com/books/opus4/html/page226.html","../www.brpreiss.com\books\opus4\html\page226.html","HTML","4130"," Sun, 09 Dec 2001 00:10:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding an Item","http://www.brpreiss.com/books/opus4/html/page227.html","../www.brpreiss.com\books\opus4\html\page227.html","HTML","3082"," Sun, 09 Dec 2001 00:10:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Case Analysis","http://www.brpreiss.com/books/opus4/html/page228.html","../www.brpreiss.com\books\opus4\html\page228.html","HTML","6229"," Sun, 09 Dec 2001 00:10:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Scatter Tables","http://www.brpreiss.com/books/opus4/html/page229.html","../www.brpreiss.com\books\opus4\html\page229.html","HTML","3126"," Sun, 09 Dec 2001 00:10:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Chained Scatter Table","http://www.brpreiss.com/books/opus4/html/page230.html","../www.brpreiss.com\books\opus4\html\page230.html","HTML","5638"," Sun, 09 Dec 2001 00:10:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page231.html","../www.brpreiss.com\books\opus4\html\page231.html","HTML","2925"," Sun, 09 Dec 2001 00:10:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors and Destructor","http://www.brpreiss.com/books/opus4/html/page232.html","../www.brpreiss.com\books\opus4\html\page232.html","HTML","4036"," Sun, 09 Dec 2001 00:10:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting and Finding an Item","http://www.brpreiss.com/books/opus4/html/page233.html","../www.brpreiss.com\books\opus4\html\page233.html","HTML","5218"," Sun, 09 Dec 2001 00:10:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items","http://www.brpreiss.com/books/opus4/html/page234.html","../www.brpreiss.com\books\opus4\html\page234.html","HTML","7296"," Sun, 09 Dec 2001 00:10:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Worst-Case Running Time","http://www.brpreiss.com/books/opus4/html/page235.html","../www.brpreiss.com\books\opus4\html\page235.html","HTML","2946"," Sun, 09 Dec 2001 00:10:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Case Analysis","http://www.brpreiss.com/books/opus4/html/page236.html","../www.brpreiss.com\books\opus4\html\page236.html","HTML","4306"," Sun, 09 Dec 2001 00:10:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Scatter Table using Open Addressing","http://www.brpreiss.com/books/opus4/html/page237.html","../www.brpreiss.com\books\opus4\html\page237.html","HTML","4541"," Sun, 09 Dec 2001 00:10:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linear Probing","http://www.brpreiss.com/books/opus4/html/page238.html","../www.brpreiss.com\books\opus4\html\page238.html","HTML","5391"," Sun, 09 Dec 2001 00:10:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Quadratic Probing","http://www.brpreiss.com/books/opus4/html/page239.html","../www.brpreiss.com\books\opus4\html\page239.html","HTML","4964"," Sun, 09 Dec 2001 00:10:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Double Hashing","http://www.brpreiss.com/books/opus4/html/page240.html","../www.brpreiss.com\books\opus4\html\page240.html","HTML","5642"," Sun, 09 Dec 2001 00:10:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page241.html","../www.brpreiss.com\books\opus4\html\page241.html","HTML","3916"," Sun, 09 Dec 2001 00:10:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors and Destructor","http://www.brpreiss.com/books/opus4/html/page242.html","../www.brpreiss.com\books\opus4\html\page242.html","HTML","3945"," Sun, 09 Dec 2001 00:24:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items","http://www.brpreiss.com/books/opus4/html/page243.html","../www.brpreiss.com\books\opus4\html\page243.html","HTML","4736"," Sun, 09 Dec 2001 00:24:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding Items","http://www.brpreiss.com/books/opus4/html/page244.html","../www.brpreiss.com\books\opus4\html\page244.html","HTML","3788"," Sun, 09 Dec 2001 00:24:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items","http://www.brpreiss.com/books/opus4/html/page245.html","../www.brpreiss.com\books\opus4\html\page245.html","HTML","10542"," Sun, 09 Dec 2001 00:24:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Case Analysis","http://www.brpreiss.com/books/opus4/html/page246.html","../www.brpreiss.com\books\opus4\html\page246.html","HTML","8250"," Sun, 09 Dec 2001 00:24:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page247.html","../www.brpreiss.com\books\opus4\html\page247.html","HTML","5391"," Sun, 09 Dec 2001 00:24:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page248.html","../www.brpreiss.com\books\opus4\html\page248.html","HTML","10024"," Sun, 09 Dec 2001 00:24:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page249.html","../www.brpreiss.com\books\opus4\html\page249.html","HTML","5064"," Sun, 09 Dec 2001 00:24:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page251.html","../www.brpreiss.com\books\opus4\html\page251.html","HTML","6543"," Sun, 09 Dec 2001 00:24:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Terminology","http://www.brpreiss.com/books/opus4/html/page252.html","../www.brpreiss.com\books\opus4\html\page252.html","HTML","5837"," Sun, 09 Dec 2001 00:24:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","More Terminology","http://www.brpreiss.com/books/opus4/html/page253.html","../www.brpreiss.com\books\opus4\html\page253.html","HTML","134223359"," Sun, 09 Dec 2001 00:24:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Alternate Representations for Trees","http://www.brpreiss.com/books/opus4/html/page254.html","../www.brpreiss.com\books\opus4\html\page254.html","HTML","3537"," Sun, 09 Dec 2001 00:25:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","N-ary Trees","http://www.brpreiss.com/books/opus4/html/page255.html","../www.brpreiss.com\books\opus4\html\page255.html","HTML","12428"," Sun, 09 Dec 2001 00:25:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Trees","http://www.brpreiss.com/books/opus4/html/page256.html","../www.brpreiss.com\books\opus4\html\page256.html","HTML","6835"," Sun, 09 Dec 2001 00:25:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tree Traversals","http://www.brpreiss.com/books/opus4/html/page257.html","../www.brpreiss.com\books\opus4\html\page257.html","HTML","4195"," Sun, 09 Dec 2001 00:25:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Preorder Traversal","http://www.brpreiss.com/books/opus4/html/page258.html","../www.brpreiss.com\books\opus4\html\page258.html","HTML","3096"," Sun, 09 Dec 2001 00:25:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Postorder Traversal","http://www.brpreiss.com/books/opus4/html/page259.html","../www.brpreiss.com\books\opus4\html\page259.html","HTML","2695"," Sun, 09 Dec 2001 00:25:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inorder Traversal","http://www.brpreiss.com/books/opus4/html/page260.html","../www.brpreiss.com\books\opus4\html\page260.html","HTML","2581"," Sun, 09 Dec 2001 00:25:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Breadth-First Traversal","http://www.brpreiss.com/books/opus4/html/page261.html","../www.brpreiss.com\books\opus4\html\page261.html","HTML","2601"," Sun, 09 Dec 2001 00:25:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Expression Trees","http://www.brpreiss.com/books/opus4/html/page262.html","../www.brpreiss.com\books\opus4\html\page262.html","HTML","5674"," Sun, 09 Dec 2001 00:25:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Infix Notation","http://www.brpreiss.com/books/opus4/html/page263.html","../www.brpreiss.com\books\opus4\html\page263.html","HTML","2710"," Sun, 09 Dec 2001 00:25:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Prefix Notation","http://www.brpreiss.com/books/opus4/html/page264.html","../www.brpreiss.com\books\opus4\html\page264.html","HTML","3067"," Sun, 09 Dec 2001 00:25:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Postfix Notation","http://www.brpreiss.com/books/opus4/html/page265.html","../www.brpreiss.com\books\opus4\html\page265.html","HTML","4175"," Sun, 09 Dec 2001 00:25:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Trees","http://www.brpreiss.com/books/opus4/html/page266.html","../www.brpreiss.com\books\opus4\html\page266.html","HTML","6230"," Sun, 09 Dec 2001 00:25:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tree Traversals","http://www.brpreiss.com/books/opus4/html/page267.html","../www.brpreiss.com\books\opus4\html\page267.html","HTML","2754"," Sun, 09 Dec 2001 00:25:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Depth-First Traversal","http://www.brpreiss.com/books/opus4/html/page268.html","../www.brpreiss.com\books\opus4\html\page268.html","HTML","3711"," Sun, 09 Dec 2001 00:25:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Preorder, Inorder and Postorder Traversals","http://www.brpreiss.com/books/opus4/html/page269.html","../www.brpreiss.com\books\opus4\html\page269.html","HTML","5040"," Sun, 09 Dec 2001 00:25:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Breadth-First Traversal","http://www.brpreiss.com/books/opus4/html/page270.html","../www.brpreiss.com\books\opus4\html\page270.html","HTML","4920"," Sun, 09 Dec 2001 00:25:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Accept Member Function","http://www.brpreiss.com/books/opus4/html/page271.html","../www.brpreiss.com\books\opus4\html\page271.html","HTML","2855"," Sun, 09 Dec 2001 00:25:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Tree Iterators","http://www.brpreiss.com/books/opus4/html/page272.html","../www.brpreiss.com\books\opus4\html\page272.html","HTML","4443"," Sun, 09 Dec 2001 00:25:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page273.html","../www.brpreiss.com\books\opus4\html\page273.html","HTML","2396"," Sun, 09 Dec 2001 00:25:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Reset Member Function","http://www.brpreiss.com/books/opus4/html/page274.html","../www.brpreiss.com\books\opus4\html\page274.html","HTML","4036"," Sun, 09 Dec 2001 00:25:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Operator Member Functions","http://www.brpreiss.com/books/opus4/html/page275.html","../www.brpreiss.com\books\opus4\html\page275.html","HTML","4002"," Sun, 09 Dec 2001 00:25:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","General Trees","http://www.brpreiss.com/books/opus4/html/page276.html","../www.brpreiss.com\books\opus4\html\page276.html","HTML","5276"," Sun, 09 Dec 2001 00:25:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page277.html","../www.brpreiss.com\books\opus4\html\page277.html","HTML","2488"," Sun, 09 Dec 2001 00:25:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Functions","http://www.brpreiss.com/books/opus4/html/page278.html","../www.brpreiss.com\books\opus4\html\page278.html","HTML","2536"," Sun, 09 Dec 2001 00:25:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor, Destructor, and Purge Member Function","http://www.brpreiss.com/books/opus4/html/page279.html","../www.brpreiss.com\books\opus4\html\page279.html","HTML","4229"," Sun, 09 Dec 2001 00:25:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Key and Subtree Member Functions","http://www.brpreiss.com/books/opus4/html/page280.html","../www.brpreiss.com\books\opus4\html\page280.html","HTML","3427"," Sun, 09 Dec 2001 00:25:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","AttachSubtree and DetachSubtree Member Functions","http://www.brpreiss.com/books/opus4/html/page281.html","../www.brpreiss.com\books\opus4\html\page281.html","HTML","3070"," Sun, 09 Dec 2001 00:25:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","N-ary Trees","http://www.brpreiss.com/books/opus4/html/page282.html","../www.brpreiss.com\books\opus4\html\page282.html","HTML","4806"," Sun, 09 Dec 2001 00:25:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page283.html","../www.brpreiss.com\books\opus4\html\page283.html","HTML","2599"," Sun, 09 Dec 2001 00:25:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Functions","http://www.brpreiss.com/books/opus4/html/page284.html","../www.brpreiss.com\books\opus4\html\page284.html","HTML","2217"," Sun, 09 Dec 2001 00:25:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors","http://www.brpreiss.com/books/opus4/html/page285.html","../www.brpreiss.com\books\opus4\html\page285.html","HTML","3365"," Sun, 09 Dec 2001 00:25:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","IsEmpty Member Function","http://www.brpreiss.com/books/opus4/html/page286.html","../www.brpreiss.com\books\opus4\html\page286.html","HTML","2622"," Sun, 09 Dec 2001 00:25:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Key, AttachKey and DetachKeyMember Functions","http://www.brpreiss.com/books/opus4/html/page287.html","../www.brpreiss.com\books\opus4\html\page287.html","HTML","3975"," Sun, 09 Dec 2001 00:25:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Subtree, AttachSubtree and DetachSubtreeMember Functions","http://www.brpreiss.com/books/opus4/html/page288.html","../www.brpreiss.com\books\opus4\html\page288.html","HTML","4087"," Sun, 09 Dec 2001 00:25:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Trees","http://www.brpreiss.com/books/opus4/html/page289.html","../www.brpreiss.com\books\opus4\html\page289.html","HTML","5255"," Sun, 09 Dec 2001 00:25:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page290.html","../www.brpreiss.com\books\opus4\html\page290.html","HTML","2278"," Sun, 09 Dec 2001 00:25:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors","http://www.brpreiss.com/books/opus4/html/page291.html","../www.brpreiss.com\books\opus4\html\page291.html","HTML","2889"," Sun, 09 Dec 2001 00:25:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Destructor and Purge Member Functions","http://www.brpreiss.com/books/opus4/html/page292.html","../www.brpreiss.com\books\opus4\html\page292.html","HTML","3524"," Sun, 09 Dec 2001 00:25:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Tree Traversals","http://www.brpreiss.com/books/opus4/html/page293.html","../www.brpreiss.com\books\opus4\html\page293.html","HTML","3158"," Sun, 09 Dec 2001 00:25:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Comparing Trees","http://www.brpreiss.com/books/opus4/html/page294.html","../www.brpreiss.com\books\opus4\html\page294.html","HTML","7989"," Sun, 09 Dec 2001 00:25:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page295.html","../www.brpreiss.com\books\opus4\html\page295.html","HTML","4199"," Sun, 09 Dec 2001 00:25:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page296.html","../www.brpreiss.com\books\opus4\html\page296.html","HTML","6056"," Sun, 09 Dec 2001 00:25:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page297.html","../www.brpreiss.com\books\opus4\html\page297.html","HTML","6550"," Sun, 09 Dec 2001 00:25:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page298.html","../www.brpreiss.com\books\opus4\html\page298.html","HTML","5762"," Sun, 09 Dec 2001 00:25:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page300.html","../www.brpreiss.com\books\opus4\html\page300.html","HTML","2794"," Sun, 09 Dec 2001 00:25:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","M-Way Search Trees","http://www.brpreiss.com/books/opus4/html/page301.html","../www.brpreiss.com\books\opus4\html\page301.html","HTML","5770"," Sun, 09 Dec 2001 00:26:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Search Trees","http://www.brpreiss.com/books/opus4/html/page302.html","../www.brpreiss.com\books\opus4\html\page302.html","HTML","4253"," Sun, 09 Dec 2001 00:26:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Searching a Search Tree","http://www.brpreiss.com/books/opus4/html/page303.html","../www.brpreiss.com\books\opus4\html\page303.html","HTML","2346"," Sun, 09 Dec 2001 00:26:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Searching an M-way Tree","http://www.brpreiss.com/books/opus4/html/page304.html","../www.brpreiss.com\books\opus4\html\page304.html","HTML","3922"," Sun, 09 Dec 2001 00:26:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Searching a Binary Tree","http://www.brpreiss.com/books/opus4/html/page305.html","../www.brpreiss.com\books\opus4\html\page305.html","HTML","5097"," Sun, 09 Dec 2001 00:26:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Case Analysis","http://www.brpreiss.com/books/opus4/html/page306.html","../www.brpreiss.com\books\opus4\html\page306.html","HTML","2283"," Sun, 09 Dec 2001 00:26:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Successful Search","http://www.brpreiss.com/books/opus4/html/page307.html","../www.brpreiss.com\books\opus4\html\page307.html","HTML","8826"," Sun, 09 Dec 2001 00:26:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Solving The Recurrence-Telescoping","http://www.brpreiss.com/books/opus4/html/page308.html","../www.brpreiss.com\books\opus4\html\page308.html","HTML","4720"," Sun, 09 Dec 2001 00:26:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Unsuccessful Search","http://www.brpreiss.com/books/opus4/html/page309.html","../www.brpreiss.com\books\opus4\html\page309.html","HTML","7831"," Sun, 09 Dec 2001 00:26:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Traversing a Search Tree","http://www.brpreiss.com/books/opus4/html/page310.html","../www.brpreiss.com\books\opus4\html\page310.html","HTML","4648"," Sun, 09 Dec 2001 00:26:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Search Trees","http://www.brpreiss.com/books/opus4/html/page311.html","../www.brpreiss.com\books\opus4\html\page311.html","HTML","4158"," Sun, 09 Dec 2001 00:26:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Search Trees","http://www.brpreiss.com/books/opus4/html/page312.html","../www.brpreiss.com\books\opus4\html\page312.html","HTML","3625"," Sun, 09 Dec 2001 00:26:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page313.html","../www.brpreiss.com\books\opus4\html\page313.html","HTML","3215"," Sun, 09 Dec 2001 00:26:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Find Member Function","http://www.brpreiss.com/books/opus4/html/page314.html","../www.brpreiss.com\books\opus4\html\page314.html","HTML","4363"," Sun, 09 Dec 2001 00:26:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","FindMin Member Function","http://www.brpreiss.com/books/opus4/html/page315.html","../www.brpreiss.com\books\opus4\html\page315.html","HTML","2832"," Sun, 09 Dec 2001 00:26:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items in a Binary Search Tree","http://www.brpreiss.com/books/opus4/html/page316.html","../www.brpreiss.com\books\opus4\html\page316.html","HTML","2476"," Sun, 09 Dec 2001 00:26:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Insert and AttachKey Member Functions","http://www.brpreiss.com/books/opus4/html/page317.html","../www.brpreiss.com\books\opus4\html\page317.html","HTML","4461"," Sun, 09 Dec 2001 00:26:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a Binary Search Tree","http://www.brpreiss.com/books/opus4/html/page318.html","../www.brpreiss.com\books\opus4\html\page318.html","HTML","4786"," Sun, 09 Dec 2001 00:26:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Withdraw and DetachKey Member Functions","http://www.brpreiss.com/books/opus4/html/page319.html","../www.brpreiss.com\books\opus4\html\page319.html","HTML","2645"," Sun, 09 Dec 2001 00:26:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","AVL Search Trees","http://www.brpreiss.com/books/opus4/html/page320.html","../www.brpreiss.com\books\opus4\html\page320.html","HTML","12395"," Sun, 09 Dec 2001 00:26:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing AVL Trees","http://www.brpreiss.com/books/opus4/html/page321.html","../www.brpreiss.com\books\opus4\html\page321.html","HTML","4247"," Sun, 09 Dec 2001 00:26:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor","http://www.brpreiss.com/books/opus4/html/page322.html","../www.brpreiss.com\books\opus4\html\page322.html","HTML","2814"," Sun, 09 Dec 2001 00:26:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Height,AdjustHeight and BalanceFactor Member Functions","http://www.brpreiss.com/books/opus4/html/page323.html","../www.brpreiss.com\books\opus4\html\page323.html","HTML","3191"," Sun, 09 Dec 2001 00:26:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items into an AVL Tree","http://www.brpreiss.com/books/opus4/html/page324.html","../www.brpreiss.com\books\opus4\html\page324.html","HTML","5730"," Sun, 09 Dec 2001 00:26:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Balancing AVL Trees","http://www.brpreiss.com/books/opus4/html/page325.html","../www.brpreiss.com\books\opus4\html\page325.html","HTML","2112"," Sun, 09 Dec 2001 00:26:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Single Rotations","http://www.brpreiss.com/books/opus4/html/page326.html","../www.brpreiss.com\books\opus4\html\page326.html","HTML","5224"," Sun, 09 Dec 2001 00:26:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Double Rotations","http://www.brpreiss.com/books/opus4/html/page327.html","../www.brpreiss.com\books\opus4\html\page327.html","HTML","6418"," Sun, 09 Dec 2001 00:26:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page328.html","../www.brpreiss.com\books\opus4\html\page328.html","HTML","5916"," Sun, 09 Dec 2001 00:26:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from an AVL Tree","http://www.brpreiss.com/books/opus4/html/page329.html","../www.brpreiss.com\books\opus4\html\page329.html","HTML","2685"," Sun, 09 Dec 2001 00:26:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","M-Way Search Trees","http://www.brpreiss.com/books/opus4/html/page330.html","../www.brpreiss.com\books\opus4\html\page330.html","HTML","5888"," Sun, 09 Dec 2001 00:26:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing M-Way Search Trees","http://www.brpreiss.com/books/opus4/html/page331.html","../www.brpreiss.com\books\opus4\html\page331.html","HTML","3356"," Sun, 09 Dec 2001 00:26:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page332.html","../www.brpreiss.com\books\opus4\html\page332.html","HTML","3707"," Sun, 09 Dec 2001 00:26:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Functions","http://www.brpreiss.com/books/opus4/html/page333.html","../www.brpreiss.com\books\opus4\html\page333.html","HTML","2913"," Sun, 09 Dec 2001 00:26:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inorder Traversal","http://www.brpreiss.com/books/opus4/html/page334.html","../www.brpreiss.com\books\opus4\html\page334.html","HTML","3560"," Sun, 09 Dec 2001 00:26:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Finding Items in an M-Way Search Tree","http://www.brpreiss.com/books/opus4/html/page335.html","../www.brpreiss.com\books\opus4\html\page335.html","HTML","2318"," Sun, 09 Dec 2001 00:26:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linear Search","http://www.brpreiss.com/books/opus4/html/page336.html","../www.brpreiss.com\books\opus4\html\page336.html","HTML","5391"," Sun, 09 Dec 2001 00:26:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Search","http://www.brpreiss.com/books/opus4/html/page337.html","../www.brpreiss.com\books\opus4\html\page337.html","HTML","5614"," Sun, 09 Dec 2001 00:26:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items into an M-Way Search Tree","http://www.brpreiss.com/books/opus4/html/page338.html","../www.brpreiss.com\books\opus4\html\page338.html","HTML","4333"," Sun, 09 Dec 2001 00:26:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from an M-Way Search Tree","http://www.brpreiss.com/books/opus4/html/page339.html","../www.brpreiss.com\books\opus4\html\page339.html","HTML","4484"," Sun, 09 Dec 2001 00:26:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","B-Trees","http://www.brpreiss.com/books/opus4/html/page340.html","../www.brpreiss.com\books\opus4\html\page340.html","HTML","7403"," Sun, 09 Dec 2001 00:26:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing B-Trees","http://www.brpreiss.com/books/opus4/html/page341.html","../www.brpreiss.com\books\opus4\html\page341.html","HTML","3121"," Sun, 09 Dec 2001 00:26:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page342.html","../www.brpreiss.com\books\opus4\html\page342.html","HTML","2406"," Sun, 09 Dec 2001 00:26:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors","http://www.brpreiss.com/books/opus4/html/page343.html","../www.brpreiss.com\books\opus4\html\page343.html","HTML","2325"," Sun, 09 Dec 2001 00:26:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Private Member Functions","http://www.brpreiss.com/books/opus4/html/page344.html","../www.brpreiss.com\books\opus4\html\page344.html","HTML","2166"," Sun, 09 Dec 2001 00:27:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inserting Items into a B-Tree","http://www.brpreiss.com/books/opus4/html/page345.html","../www.brpreiss.com\books\opus4\html\page345.html","HTML","8267"," Sun, 09 Dec 2001 00:27:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page346.html","../www.brpreiss.com\books\opus4\html\page346.html","HTML","7573"," Sun, 09 Dec 2001 00:27:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page347.html","../www.brpreiss.com\books\opus4\html\page347.html","HTML","3590"," Sun, 09 Dec 2001 00:27:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a B-Tree","http://www.brpreiss.com/books/opus4/html/page348.html","../www.brpreiss.com\books\opus4\html\page348.html","HTML","6098"," Sun, 09 Dec 2001 00:27:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page349.html","../www.brpreiss.com\books\opus4\html\page349.html","HTML","5052"," Sun, 09 Dec 2001 00:27:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page350.html","../www.brpreiss.com\books\opus4\html\page350.html","HTML","8380"," Sun, 09 Dec 2001 00:27:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page351.html","../www.brpreiss.com\books\opus4\html\page351.html","HTML","6227"," Sun, 09 Dec 2001 00:27:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page353.html","../www.brpreiss.com\books\opus4\html\page353.html","HTML","6974"," Sun, 09 Dec 2001 00:27:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Heaps","http://www.brpreiss.com/books/opus4/html/page354.html","../www.brpreiss.com\books\opus4\html\page354.html","HTML","2775"," Sun, 09 Dec 2001 00:27:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Complete Trees","http://www.brpreiss.com/books/opus4/html/page355.html","../www.brpreiss.com\books\opus4\html\page355.html","HTML","12915"," Sun, 09 Dec 2001 00:27:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Complete N-ary Trees","http://www.brpreiss.com/books/opus4/html/page356.html","../www.brpreiss.com\books\opus4\html\page356.html","HTML","6779"," Sun, 09 Dec 2001 00:27:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page357.html","../www.brpreiss.com\books\opus4\html\page357.html","HTML","3203"," Sun, 09 Dec 2001 00:27:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page358.html","../www.brpreiss.com\books\opus4\html\page358.html","HTML","2252"," Sun, 09 Dec 2001 00:27:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor, Destructor and Purge Member Functions","http://www.brpreiss.com/books/opus4/html/page359.html","../www.brpreiss.com\books\opus4\html\page359.html","HTML","3548"," Sun, 09 Dec 2001 00:27:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Putting Items into a Binary Heap","http://www.brpreiss.com/books/opus4/html/page360.html","../www.brpreiss.com\books\opus4\html\page360.html","HTML","5571"," Sun, 09 Dec 2001 00:27:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a Binary Heap","http://www.brpreiss.com/books/opus4/html/page361.html","../www.brpreiss.com\books\opus4\html\page361.html","HTML","6442"," Sun, 09 Dec 2001 00:27:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Leftist Heaps","http://www.brpreiss.com/books/opus4/html/page362.html","../www.brpreiss.com\books\opus4\html\page362.html","HTML","2718"," Sun, 09 Dec 2001 00:27:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Leftist Trees","http://www.brpreiss.com/books/opus4/html/page363.html","../www.brpreiss.com\books\opus4\html\page363.html","HTML","8824"," Sun, 09 Dec 2001 00:27:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page364.html","../www.brpreiss.com\books\opus4\html\page364.html","HTML","2678"," Sun, 09 Dec 2001 00:27:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page365.html","../www.brpreiss.com\books\opus4\html\page365.html","HTML","2438"," Sun, 09 Dec 2001 00:27:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","SwapContents Member Function","http://www.brpreiss.com/books/opus4/html/page366.html","../www.brpreiss.com\books\opus4\html\page366.html","HTML","2389"," Sun, 09 Dec 2001 00:27:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Merging Leftist Heaps","http://www.brpreiss.com/books/opus4/html/page367.html","../www.brpreiss.com\books\opus4\html\page367.html","HTML","6687"," Sun, 09 Dec 2001 00:27:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Putting Items into a Leftist Heap","http://www.brpreiss.com/books/opus4/html/page368.html","../www.brpreiss.com\books\opus4\html\page368.html","HTML","3275"," Sun, 09 Dec 2001 00:27:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing Items from a Leftist Heap","http://www.brpreiss.com/books/opus4/html/page369.html","../www.brpreiss.com\books\opus4\html\page369.html","HTML","4826"," Sun, 09 Dec 2001 00:27:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binomial Queues","http://www.brpreiss.com/books/opus4/html/page370.html","../www.brpreiss.com\books\opus4\html\page370.html","HTML","3322"," Sun, 09 Dec 2001 00:27:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binomial Trees","http://www.brpreiss.com/books/opus4/html/page371.html","../www.brpreiss.com\books\opus4\html\page371.html","HTML","15137"," Sun, 09 Dec 2001 00:27:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binomial Queues","http://www.brpreiss.com/books/opus4/html/page372.html","../www.brpreiss.com\books\opus4\html\page372.html","HTML","9880"," Sun, 09 Dec 2001 00:27:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page373.html","../www.brpreiss.com\books\opus4\html\page373.html","HTML","2414"," Sun, 09 Dec 2001 00:27:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Heap-Ordered Binomial Trees","http://www.brpreiss.com/books/opus4/html/page374.html","../www.brpreiss.com\books\opus4\html\page374.html","HTML","5858"," Sun, 09 Dec 2001 00:27:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binomial Queues","http://www.brpreiss.com/books/opus4/html/page375.html","../www.brpreiss.com\books\opus4\html\page375.html","HTML","2554"," Sun, 09 Dec 2001 00:27:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables","http://www.brpreiss.com/books/opus4/html/page376.html","../www.brpreiss.com\books\opus4\html\page376.html","HTML","2228"," Sun, 09 Dec 2001 00:27:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","AddTree and RemoveTree","http://www.brpreiss.com/books/opus4/html/page377.html","../www.brpreiss.com\books\opus4\html\page377.html","HTML","3606"," Sun, 09 Dec 2001 00:27:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","FindMinTree and FindMin Member Functions","http://www.brpreiss.com/books/opus4/html/page378.html","../www.brpreiss.com\books\opus4\html\page378.html","HTML","3748"," Sun, 09 Dec 2001 00:27:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Merging Binomial Queues","http://www.brpreiss.com/books/opus4/html/page379.html","../www.brpreiss.com\books\opus4\html\page379.html","HTML","9420"," Sun, 09 Dec 2001 00:27:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Putting Items into a Binomial Queue","http://www.brpreiss.com/books/opus4/html/page380.html","../www.brpreiss.com\books\opus4\html\page380.html","HTML","3024"," Sun, 09 Dec 2001 00:27:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Removing an Item from a Binomial Queue","http://www.brpreiss.com/books/opus4/html/page381.html","../www.brpreiss.com\books\opus4\html\page381.html","HTML","4731"," Sun, 09 Dec 2001 00:27:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page382.html","../www.brpreiss.com\books\opus4\html\page382.html","HTML","2044"," Sun, 09 Dec 2001 00:27:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Discrete Event Simulation","http://www.brpreiss.com/books/opus4/html/page383.html","../www.brpreiss.com\books\opus4\html\page383.html","HTML","5046"," Sun, 09 Dec 2001 00:27:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page384.html","../www.brpreiss.com\books\opus4\html\page384.html","HTML","7469"," Sun, 09 Dec 2001 00:27:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page385.html","../www.brpreiss.com\books\opus4\html\page385.html","HTML","6969"," Sun, 09 Dec 2001 00:28:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page386.html","../www.brpreiss.com\books\opus4\html\page386.html","HTML","6426"," Sun, 09 Dec 2001 00:28:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page388.html","../www.brpreiss.com\books\opus4\html\page388.html","HTML","2839"," Sun, 09 Dec 2001 00:28:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Sets","http://www.brpreiss.com/books/opus4/html/page389.html","../www.brpreiss.com\books\opus4\html\page389.html","HTML","4991"," Sun, 09 Dec 2001 00:28:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array and Bit-Vector Sets","http://www.brpreiss.com/books/opus4/html/page390.html","../www.brpreiss.com\books\opus4\html\page390.html","HTML","5314"," Sun, 09 Dec 2001 00:28:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basic Operations","http://www.brpreiss.com/books/opus4/html/page391.html","../www.brpreiss.com\books\opus4\html\page391.html","HTML","4475"," Sun, 09 Dec 2001 00:28:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union, Intersection and Difference","http://www.brpreiss.com/books/opus4/html/page392.html","../www.brpreiss.com\books\opus4\html\page392.html","HTML","4471"," Sun, 09 Dec 2001 00:28:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Comparing Sets","http://www.brpreiss.com/books/opus4/html/page393.html","../www.brpreiss.com\books\opus4\html\page393.html","HTML","5718"," Sun, 09 Dec 2001 00:28:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Bit-Vector Sets","http://www.brpreiss.com/books/opus4/html/page394.html","../www.brpreiss.com\books\opus4\html\page394.html","HTML","3481"," Sun, 09 Dec 2001 00:28:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basic Operations","http://www.brpreiss.com/books/opus4/html/page395.html","../www.brpreiss.com\books\opus4\html\page395.html","HTML","5072"," Sun, 09 Dec 2001 00:28:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union, Intersection and Difference","http://www.brpreiss.com/books/opus4/html/page396.html","../www.brpreiss.com\books\opus4\html\page396.html","HTML","3709"," Sun, 09 Dec 2001 00:28:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Multisets","http://www.brpreiss.com/books/opus4/html/page397.html","../www.brpreiss.com\books\opus4\html\page397.html","HTML","3032"," Sun, 09 Dec 2001 00:28:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Array Implementation","http://www.brpreiss.com/books/opus4/html/page398.html","../www.brpreiss.com\books\opus4\html\page398.html","HTML","3086"," Sun, 09 Dec 2001 00:28:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basic Operations","http://www.brpreiss.com/books/opus4/html/page399.html","../www.brpreiss.com\books\opus4\html\page399.html","HTML","2960"," Sun, 09 Dec 2001 00:28:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union, Intersection and Difference","http://www.brpreiss.com/books/opus4/html/page400.html","../www.brpreiss.com\books\opus4\html\page400.html","HTML","6757"," Sun, 09 Dec 2001 00:28:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Linked List Implementation","http://www.brpreiss.com/books/opus4/html/page401.html","../www.brpreiss.com\books\opus4\html\page401.html","HTML","4695"," Sun, 09 Dec 2001 00:28:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union","http://www.brpreiss.com/books/opus4/html/page402.html","../www.brpreiss.com\books\opus4\html\page402.html","HTML","3199"," Sun, 09 Dec 2001 00:28:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Intersection","http://www.brpreiss.com/books/opus4/html/page403.html","../www.brpreiss.com\books\opus4\html\page403.html","HTML","3416"," Sun, 09 Dec 2001 00:28:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Partitions","http://www.brpreiss.com/books/opus4/html/page404.html","../www.brpreiss.com\books\opus4\html\page404.html","HTML","7386"," Sun, 09 Dec 2001 00:28:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Representing Partitions","http://www.brpreiss.com/books/opus4/html/page405.html","../www.brpreiss.com\books\opus4\html\page405.html","HTML","3320"," Sun, 09 Dec 2001 00:28:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing a Partition using a Forest","http://www.brpreiss.com/books/opus4/html/page406.html","../www.brpreiss.com\books\opus4\html\page406.html","HTML","5044"," Sun, 09 Dec 2001 00:28:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page407.html","../www.brpreiss.com\books\opus4\html\page407.html","HTML","4220"," Sun, 09 Dec 2001 00:28:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors and Destructor","http://www.brpreiss.com/books/opus4/html/page408.html","../www.brpreiss.com\books\opus4\html\page408.html","HTML","3717"," Sun, 09 Dec 2001 00:28:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Find and Join Member Functions","http://www.brpreiss.com/books/opus4/html/page409.html","../www.brpreiss.com\books\opus4\html\page409.html","HTML","6926"," Sun, 09 Dec 2001 00:28:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Collapsing Find","http://www.brpreiss.com/books/opus4/html/page410.html","../www.brpreiss.com\books\opus4\html\page410.html","HTML","5402"," Sun, 09 Dec 2001 00:28:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union by Size","http://www.brpreiss.com/books/opus4/html/page411.html","../www.brpreiss.com\books\opus4\html\page411.html","HTML","7800"," Sun, 09 Dec 2001 00:28:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Union by Height or Rank","http://www.brpreiss.com/books/opus4/html/page412.html","../www.brpreiss.com\books\opus4\html\page412.html","HTML","4382"," Sun, 09 Dec 2001 00:28:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page413.html","../www.brpreiss.com\books\opus4\html\page413.html","HTML","8739"," Sun, 09 Dec 2001 00:28:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page414.html","../www.brpreiss.com\books\opus4\html\page414.html","HTML","8643"," Sun, 09 Dec 2001 00:28:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page415.html","../www.brpreiss.com\books\opus4\html\page415.html","HTML","5163"," Sun, 09 Dec 2001 00:28:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page417.html","../www.brpreiss.com\books\opus4\html\page417.html","HTML","5004"," Sun, 09 Dec 2001 00:28:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","C++ Magic","http://www.brpreiss.com/books/opus4/html/page418.html","../www.brpreiss.com\books\opus4\html\page418.html","HTML","4857"," Sun, 09 Dec 2001 00:28:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Working with Multiple Storage Pools","http://www.brpreiss.com/books/opus4/html/page419.html","../www.brpreiss.com\books\opus4\html\page419.html","HTML","7276"," Sun, 09 Dec 2001 00:28:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Heap","http://www.brpreiss.com/books/opus4/html/page420.html","../www.brpreiss.com\books\opus4\html\page420.html","HTML","3396"," Sun, 09 Dec 2001 00:28:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Singly Linked Free Storage","http://www.brpreiss.com/books/opus4/html/page421.html","../www.brpreiss.com\books\opus4\html\page421.html","HTML","5054"," Sun, 09 Dec 2001 00:28:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page422.html","../www.brpreiss.com\books\opus4\html\page422.html","HTML","4577"," Sun, 09 Dec 2001 00:28:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page423.html","../www.brpreiss.com\books\opus4\html\page423.html","HTML","5011"," Sun, 09 Dec 2001 00:28:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Acquiring an Area","http://www.brpreiss.com/books/opus4/html/page424.html","../www.brpreiss.com\books\opus4\html\page424.html","HTML","5920"," Sun, 09 Dec 2001 00:28:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Releasing an Area","http://www.brpreiss.com/books/opus4/html/page425.html","../www.brpreiss.com\books\opus4\html\page425.html","HTML","7315"," Sun, 09 Dec 2001 00:28:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Doubly Linked Free Storage","http://www.brpreiss.com/books/opus4/html/page426.html","../www.brpreiss.com\books\opus4\html\page426.html","HTML","6105"," Sun, 09 Dec 2001 00:28:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page427.html","../www.brpreiss.com\books\opus4\html\page427.html","HTML","4443"," Sun, 09 Dec 2001 00:28:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page428.html","../www.brpreiss.com\books\opus4\html\page428.html","HTML","4572"," Sun, 09 Dec 2001 00:28:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Releasing an Area","http://www.brpreiss.com/books/opus4/html/page429.html","../www.brpreiss.com\books\opus4\html\page429.html","HTML","3004"," Sun, 09 Dec 2001 00:28:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Acquiring an Area","http://www.brpreiss.com/books/opus4/html/page430.html","../www.brpreiss.com\books\opus4\html\page430.html","HTML","7585"," Sun, 09 Dec 2001 00:28:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Buddy System for Storage Management","http://www.brpreiss.com/books/opus4/html/page431.html","../www.brpreiss.com\books\opus4\html\page431.html","HTML","10332"," Sun, 09 Dec 2001 00:28:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page432.html","../www.brpreiss.com\books\opus4\html\page432.html","HTML","4252"," Sun, 09 Dec 2001 00:29:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructor and Destructor","http://www.brpreiss.com/books/opus4/html/page433.html","../www.brpreiss.com\books\opus4\html\page433.html","HTML","5220"," Sun, 09 Dec 2001 00:29:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Acquiring an Area","http://www.brpreiss.com/books/opus4/html/page434.html","../www.brpreiss.com\books\opus4\html\page434.html","HTML","6030"," Sun, 09 Dec 2001 00:29:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Releasing an Area","http://www.brpreiss.com/books/opus4/html/page435.html","../www.brpreiss.com\books\opus4\html\page435.html","HTML","7091"," Sun, 09 Dec 2001 00:29:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Applications","http://www.brpreiss.com/books/opus4/html/page436.html","../www.brpreiss.com\books\opus4\html\page436.html","HTML","5049"," Sun, 09 Dec 2001 00:29:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page437.html","../www.brpreiss.com\books\opus4\html\page437.html","HTML","6271"," Sun, 09 Dec 2001 00:29:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page438.html","../www.brpreiss.com\books\opus4\html\page438.html","HTML","6010"," Sun, 09 Dec 2001 00:29:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page439.html","../www.brpreiss.com\books\opus4\html\page439.html","HTML","5807"," Sun, 09 Dec 2001 00:29:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Brute-Force and Greedy Algorithms","http://www.brpreiss.com/books/opus4/html/page441.html","../www.brpreiss.com\books\opus4\html\page441.html","HTML","3716"," Sun, 09 Dec 2001 00:29:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Counting Change","http://www.brpreiss.com/books/opus4/html/page442.html","../www.brpreiss.com\books\opus4\html\page442.html","HTML","3882"," Sun, 09 Dec 2001 00:29:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Brute-Force Algorithm","http://www.brpreiss.com/books/opus4/html/page443.html","../www.brpreiss.com\books\opus4\html\page443.html","HTML","3374"," Sun, 09 Dec 2001 00:29:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Greedy Algorithm","http://www.brpreiss.com/books/opus4/html/page444.html","../www.brpreiss.com\books\opus4\html\page444.html","HTML","3341"," Sun, 09 Dec 2001 00:29:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-0/1 Knapsack Problem","http://www.brpreiss.com/books/opus4/html/page445.html","../www.brpreiss.com\books\opus4\html\page445.html","HTML","10332"," Sun, 09 Dec 2001 00:29:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Backtracking Algorithms","http://www.brpreiss.com/books/opus4/html/page446.html","../www.brpreiss.com\books\opus4\html\page446.html","HTML","3047"," Sun, 09 Dec 2001 00:29:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Balancing Scales","http://www.brpreiss.com/books/opus4/html/page447.html","../www.brpreiss.com\books\opus4\html\page447.html","HTML","6662"," Sun, 09 Dec 2001 00:29:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Representing the Solution Space","http://www.brpreiss.com/books/opus4/html/page448.html","../www.brpreiss.com\books\opus4\html\page448.html","HTML","4954"," Sun, 09 Dec 2001 00:29:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstract Backtracking Solvers","http://www.brpreiss.com/books/opus4/html/page449.html","../www.brpreiss.com\books\opus4\html\page449.html","HTML","5034"," Sun, 09 Dec 2001 00:29:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Depth-First Solver","http://www.brpreiss.com/books/opus4/html/page450.html","../www.brpreiss.com\books\opus4\html\page450.html","HTML","3798"," Sun, 09 Dec 2001 00:29:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Breadth-First Solver","http://www.brpreiss.com/books/opus4/html/page451.html","../www.brpreiss.com\books\opus4\html\page451.html","HTML","3941"," Sun, 09 Dec 2001 00:29:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Branch-and-Bound Solvers","http://www.brpreiss.com/books/opus4/html/page452.html","../www.brpreiss.com\books\opus4\html\page452.html","HTML","5006"," Sun, 09 Dec 2001 00:29:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Depth-First, Branch-and-Bound Solver","http://www.brpreiss.com/books/opus4/html/page453.html","../www.brpreiss.com\books\opus4\html\page453.html","HTML","4134"," Sun, 09 Dec 2001 00:29:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-0/1 Knapsack Problem Again","http://www.brpreiss.com/books/opus4/html/page454.html","../www.brpreiss.com\books\opus4\html\page454.html","HTML","6164"," Sun, 09 Dec 2001 00:29:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Top-Down Algorithms: Divide-and-Conquer","http://www.brpreiss.com/books/opus4/html/page455.html","../www.brpreiss.com\books\opus4\html\page455.html","HTML","3129"," Sun, 09 Dec 2001 00:29:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Binary Search","http://www.brpreiss.com/books/opus4/html/page456.html","../www.brpreiss.com\books\opus4\html\page456.html","HTML","4740"," Sun, 09 Dec 2001 00:29:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Computing Fibonacci Numbers","http://www.brpreiss.com/books/opus4/html/page457.html","../www.brpreiss.com\books\opus4\html\page457.html","HTML","5563"," Sun, 09 Dec 2001 00:29:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Merge Sorting","http://www.brpreiss.com/books/opus4/html/page458.html","../www.brpreiss.com\books\opus4\html\page458.html","HTML","4964"," Sun, 09 Dec 2001 00:29:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time of Divide-and-Conquer Algorithms","http://www.brpreiss.com/books/opus4/html/page459.html","../www.brpreiss.com\books\opus4\html\page459.html","HTML","9387"," Sun, 09 Dec 2001 00:29:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Case 1 (#tex2html_wrap_inline68825#)","http://www.brpreiss.com/books/opus4/html/page460.html","../www.brpreiss.com\books\opus4\html\page460.html","HTML","3093"," Sun, 09 Dec 2001 00:29:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Saved",")","http://www.brpreiss.com/books/opus4/html/img1902.gif","../www.brpreiss.com\books\opus4\html\img1902.gif","Images","131"," Sun, 09 Dec 2001 00:47:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Case 2 (#tex2html_wrap_inline68839#)","http://www.brpreiss.com/books/opus4/html/page461.html","../www.brpreiss.com\books\opus4\html\page461.html","HTML","2769"," Sun, 09 Dec 2001 00:29:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Saved",")","http://www.brpreiss.com/books/opus4/html/img1908.gif","../www.brpreiss.com\books\opus4\html\img1908.gif","Images","106"," Sun, 09 Dec 2001 00:47:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Case 3 (#tex2html_wrap_inline68849#)","http://www.brpreiss.com/books/opus4/html/page462.html","../www.brpreiss.com\books\opus4\html\page462.html","HTML","2647"," Sun, 09 Dec 2001 00:29:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Saved",")","http://www.brpreiss.com/books/opus4/html/img1911.gif","../www.brpreiss.com\books\opus4\html\img1911.gif","Images","131"," Sun, 09 Dec 2001 00:47:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Summary","http://www.brpreiss.com/books/opus4/html/page463.html","../www.brpreiss.com\books\opus4\html\page463.html","HTML","6074"," Sun, 09 Dec 2001 00:29:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Matrix Multiplication","http://www.brpreiss.com/books/opus4/html/page464.html","../www.brpreiss.com\books\opus4\html\page464.html","HTML","8854"," Sun, 09 Dec 2001 00:29:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Bottom-Up Algorithms: Dynamic Programming","http://www.brpreiss.com/books/opus4/html/page465.html","../www.brpreiss.com\books\opus4\html\page465.html","HTML","2752"," Sun, 09 Dec 2001 00:29:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Generalized Fibonacci Numbers","http://www.brpreiss.com/books/opus4/html/page466.html","../www.brpreiss.com\books\opus4\html\page466.html","HTML","5610"," Sun, 09 Dec 2001 00:29:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Computing Binomial Coefficients","http://www.brpreiss.com/books/opus4/html/page467.html","../www.brpreiss.com\books\opus4\html\page467.html","HTML","12530"," Sun, 09 Dec 2001 00:29:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Application: Typesetting Problem","http://www.brpreiss.com/books/opus4/html/page468.html","../www.brpreiss.com\books\opus4\html\page468.html","HTML","7822"," Sun, 09 Dec 2001 00:29:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example","http://www.brpreiss.com/books/opus4/html/page469.html","../www.brpreiss.com\books\opus4\html\page469.html","HTML","14150"," Sun, 09 Dec 2001 00:29:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page470.html","../www.brpreiss.com\books\opus4\html\page470.html","HTML","4479"," Sun, 09 Dec 2001 00:29:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Randomized Algorithms","http://www.brpreiss.com/books/opus4/html/page471.html","../www.brpreiss.com\books\opus4\html\page471.html","HTML","3211"," Sun, 09 Dec 2001 00:29:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Generating Random Numbers","http://www.brpreiss.com/books/opus4/html/page472.html","../www.brpreiss.com\books\opus4\html\page472.html","HTML","7845"," Sun, 09 Dec 2001 00:29:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Minimal Standard Random Number Generator","http://www.brpreiss.com/books/opus4/html/page473.html","../www.brpreiss.com\books\opus4\html\page473.html","HTML","6243"," Sun, 09 Dec 2001 00:29:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page474.html","../www.brpreiss.com\books\opus4\html\page474.html","HTML","3864"," Sun, 09 Dec 2001 00:29:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Random Variables","http://www.brpreiss.com/books/opus4/html/page475.html","../www.brpreiss.com\books\opus4\html\page475.html","HTML","4161"," Sun, 09 Dec 2001 00:29:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page476.html","../www.brpreiss.com\books\opus4\html\page476.html","HTML","5392"," Sun, 09 Dec 2001 00:29:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Monte Carlo Methods","http://www.brpreiss.com/books/opus4/html/page477.html","../www.brpreiss.com\books\opus4\html\page477.html","HTML","3230"," Sun, 09 Dec 2001 00:29:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Computing #tex2html_wrap_inline69477#","http://www.brpreiss.com/books/opus4/html/page478.html","../www.brpreiss.com\books\opus4\html\page478.html","HTML","5323"," Sun, 09 Dec 2001 00:29:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img2047.gif","../www.brpreiss.com\books\opus4\html\img2047.gif","Images","55"," Sun, 09 Dec 2001 00:50:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Simulated Annealing","http://www.brpreiss.com/books/opus4/html/page479.html","../www.brpreiss.com\books\opus4\html\page479.html","HTML","5938"," Sun, 09 Dec 2001 00:30:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Example-Balancing Scales","http://www.brpreiss.com/books/opus4/html/page480.html","../www.brpreiss.com\books\opus4\html\page480.html","HTML","3413"," Sun, 09 Dec 2001 00:30:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page481.html","../www.brpreiss.com\books\opus4\html\page481.html","HTML","12085"," Sun, 09 Dec 2001 00:30:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page482.html","../www.brpreiss.com\books\opus4\html\page482.html","HTML","7488"," Sun, 09 Dec 2001 00:30:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page484.html","../www.brpreiss.com\books\opus4\html\page484.html","HTML","5529"," Sun, 09 Dec 2001 00:30:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Sorting and Sorters","http://www.brpreiss.com/books/opus4/html/page485.html","../www.brpreiss.com\books\opus4\html\page485.html","HTML","5763"," Sun, 09 Dec 2001 00:30:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Sorter Class Hierarchy","http://www.brpreiss.com/books/opus4/html/page486.html","../www.brpreiss.com\books\opus4\html\page486.html","HTML","3279"," Sun, 09 Dec 2001 00:30:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Insertion Sorting","http://www.brpreiss.com/books/opus4/html/page487.html","../www.brpreiss.com\books\opus4\html\page487.html","HTML","5875"," Sun, 09 Dec 2001 00:30:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Straight Insertion Sort","http://www.brpreiss.com/books/opus4/html/page488.html","../www.brpreiss.com\books\opus4\html\page488.html","HTML","2541"," Sun, 09 Dec 2001 00:30:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page489.html","../www.brpreiss.com\books\opus4\html\page489.html","HTML","3970"," Sun, 09 Dec 2001 00:30:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Running Time","http://www.brpreiss.com/books/opus4/html/page490.html","../www.brpreiss.com\books\opus4\html\page490.html","HTML","6638"," Sun, 09 Dec 2001 00:30:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Binary Insertion Sort","http://www.brpreiss.com/books/opus4/html/page491.html","../www.brpreiss.com\books\opus4\html\page491.html","HTML","7730"," Sun, 09 Dec 2001 00:30:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exchange Sorting","http://www.brpreiss.com/books/opus4/html/page492.html","../www.brpreiss.com\books\opus4\html\page492.html","HTML","3059"," Sun, 09 Dec 2001 00:30:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Bubble Sort","http://www.brpreiss.com/books/opus4/html/page493.html","../www.brpreiss.com\books\opus4\html\page493.html","HTML","6413"," Sun, 09 Dec 2001 00:30:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Quicksort","http://www.brpreiss.com/books/opus4/html/page494.html","../www.brpreiss.com\books\opus4\html\page494.html","HTML","7267"," Sun, 09 Dec 2001 00:30:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page495.html","../www.brpreiss.com\books\opus4\html\page495.html","HTML","6809"," Sun, 09 Dec 2001 00:30:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page496.html","../www.brpreiss.com\books\opus4\html\page496.html","HTML","4069"," Sun, 09 Dec 2001 00:30:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Worst-Case Running Time","http://www.brpreiss.com/books/opus4/html/page497.html","../www.brpreiss.com\books\opus4\html\page497.html","HTML","2438"," Sun, 09 Dec 2001 00:30:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Best-Case Running Time","http://www.brpreiss.com/books/opus4/html/page498.html","../www.brpreiss.com\books\opus4\html\page498.html","HTML","2636"," Sun, 09 Dec 2001 00:30:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Average Running Time","http://www.brpreiss.com/books/opus4/html/page499.html","../www.brpreiss.com\books\opus4\html\page499.html","HTML","7482"," Sun, 09 Dec 2001 00:30:27 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Selecting the Pivot","http://www.brpreiss.com/books/opus4/html/page500.html","../www.brpreiss.com\books\opus4\html\page500.html","HTML","5765"," Sun, 09 Dec 2001 00:30:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Selection Sorting","http://www.brpreiss.com/books/opus4/html/page501.html","../www.brpreiss.com\books\opus4\html\page501.html","HTML","3414"," Sun, 09 Dec 2001 00:30:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Straight Selection Sorting","http://www.brpreiss.com/books/opus4/html/page502.html","../www.brpreiss.com\books\opus4\html\page502.html","HTML","3475"," Sun, 09 Dec 2001 00:30:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page503.html","../www.brpreiss.com\books\opus4\html\page503.html","HTML","3920"," Sun, 09 Dec 2001 00:30:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Sorting with a Heap","http://www.brpreiss.com/books/opus4/html/page504.html","../www.brpreiss.com\books\opus4\html\page504.html","HTML","4756"," Sun, 09 Dec 2001 00:30:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page505.html","../www.brpreiss.com\books\opus4\html\page505.html","HTML","7228"," Sun, 09 Dec 2001 00:30:34 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Building the Heap","http://www.brpreiss.com/books/opus4/html/page506.html","../www.brpreiss.com\books\opus4\html\page506.html","HTML","3938"," Sun, 09 Dec 2001 00:30:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page507.html","../www.brpreiss.com\books\opus4\html\page507.html","HTML","5324"," Sun, 09 Dec 2001 00:30:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Sorting Phase","http://www.brpreiss.com/books/opus4/html/page508.html","../www.brpreiss.com\books\opus4\html\page508.html","HTML","5174"," Sun, 09 Dec 2001 00:30:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Merge Sorting","http://www.brpreiss.com/books/opus4/html/page509.html","../www.brpreiss.com\books\opus4\html\page509.html","HTML","4241"," Sun, 09 Dec 2001 00:30:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page510.html","../www.brpreiss.com\books\opus4\html\page510.html","HTML","2923"," Sun, 09 Dec 2001 00:30:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Merging","http://www.brpreiss.com/books/opus4/html/page511.html","../www.brpreiss.com\books\opus4\html\page511.html","HTML","3835"," Sun, 09 Dec 2001 00:30:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Two-Way Merge Sorting","http://www.brpreiss.com/books/opus4/html/page512.html","../www.brpreiss.com\books\opus4\html\page512.html","HTML","3306"," Sun, 09 Dec 2001 00:30:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page513.html","../www.brpreiss.com\books\opus4\html\page513.html","HTML","3195"," Sun, 09 Dec 2001 00:30:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","A Lower Bound on Sorting","http://www.brpreiss.com/books/opus4/html/page514.html","../www.brpreiss.com\books\opus4\html\page514.html","HTML","6179"," Sun, 09 Dec 2001 00:30:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Distribution Sorting","http://www.brpreiss.com/books/opus4/html/page515.html","../www.brpreiss.com\books\opus4\html\page515.html","HTML","3127"," Sun, 09 Dec 2001 00:30:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Bucket Sort","http://www.brpreiss.com/books/opus4/html/page516.html","../www.brpreiss.com\books\opus4\html\page516.html","HTML","3649"," Sun, 09 Dec 2001 00:30:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page517.html","../www.brpreiss.com\books\opus4\html\page517.html","HTML","5393"," Sun, 09 Dec 2001 00:30:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Radix Sort","http://www.brpreiss.com/books/opus4/html/page518.html","../www.brpreiss.com\books\opus4\html\page518.html","HTML","9365"," Sun, 09 Dec 2001 00:30:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page519.html","../www.brpreiss.com\books\opus4\html\page519.html","HTML","6097"," Sun, 09 Dec 2001 00:30:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Performance Data","http://www.brpreiss.com/books/opus4/html/page520.html","../www.brpreiss.com\books\opus4\html\page520.html","HTML","7347"," Sun, 09 Dec 2001 00:30:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page521.html","../www.brpreiss.com\books\opus4\html\page521.html","HTML","9875"," Sun, 09 Dec 2001 00:30:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page522.html","../www.brpreiss.com\books\opus4\html\page522.html","HTML","4078"," Sun, 09 Dec 2001 00:30:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Basics","http://www.brpreiss.com/books/opus4/html/page524.html","../www.brpreiss.com\books\opus4\html\page524.html","HTML","2617"," Sun, 09 Dec 2001 00:30:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Directed Graphs","http://www.brpreiss.com/books/opus4/html/page525.html","../www.brpreiss.com\books\opus4\html\page525.html","HTML","4162"," Sun, 09 Dec 2001 00:30:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Terminology","http://www.brpreiss.com/books/opus4/html/page526.html","../www.brpreiss.com\books\opus4\html\page526.html","HTML","10372"," Sun, 09 Dec 2001 00:31:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","More Terminology","http://www.brpreiss.com/books/opus4/html/page527.html","../www.brpreiss.com\books\opus4\html\page527.html","HTML","5202"," Sun, 09 Dec 2001 00:31:01 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Directed Acyclic Graphs","http://www.brpreiss.com/books/opus4/html/page528.html","../www.brpreiss.com\books\opus4\html\page528.html","HTML","3235"," Sun, 09 Dec 2001 00:31:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Undirected Graphs","http://www.brpreiss.com/books/opus4/html/page529.html","../www.brpreiss.com\books\opus4\html\page529.html","HTML","4769"," Sun, 09 Dec 2001 00:31:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Terminology","http://www.brpreiss.com/books/opus4/html/page530.html","../www.brpreiss.com\books\opus4\html\page530.html","HTML","2572"," Sun, 09 Dec 2001 00:31:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Labeled Graphs","http://www.brpreiss.com/books/opus4/html/page531.html","../www.brpreiss.com\books\opus4\html\page531.html","HTML","3586"," Sun, 09 Dec 2001 00:31:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Representing Graphs","http://www.brpreiss.com/books/opus4/html/page532.html","../www.brpreiss.com\books\opus4\html\page532.html","HTML","2882"," Sun, 09 Dec 2001 00:31:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Adjacency Matrices","http://www.brpreiss.com/books/opus4/html/page533.html","../www.brpreiss.com\books\opus4\html\page533.html","HTML","8004"," Sun, 09 Dec 2001 00:31:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Sparse vs. Dense Graphs","http://www.brpreiss.com/books/opus4/html/page534.html","../www.brpreiss.com\books\opus4\html\page534.html","HTML","3753"," Sun, 09 Dec 2001 00:31:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Adjacency Lists","http://www.brpreiss.com/books/opus4/html/page535.html","../www.brpreiss.com\books\opus4\html\page535.html","HTML","4320"," Sun, 09 Dec 2001 00:31:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Graphs","http://www.brpreiss.com/books/opus4/html/page536.html","../www.brpreiss.com\books\opus4\html\page536.html","HTML","3437"," Sun, 09 Dec 2001 00:31:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Vertices","http://www.brpreiss.com/books/opus4/html/page537.html","../www.brpreiss.com\books\opus4\html\page537.html","HTML","3179"," Sun, 09 Dec 2001 00:31:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Edges","http://www.brpreiss.com/books/opus4/html/page538.html","../www.brpreiss.com\books\opus4\html\page538.html","HTML","4133"," Sun, 09 Dec 2001 00:31:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstract Graphs and Digraphs","http://www.brpreiss.com/books/opus4/html/page539.html","../www.brpreiss.com\books\opus4\html\page539.html","HTML","5198"," Sun, 09 Dec 2001 00:31:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Accessors and Mutators","http://www.brpreiss.com/books/opus4/html/page540.html","../www.brpreiss.com\books\opus4\html\page540.html","HTML","4474"," Sun, 09 Dec 2001 00:31:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Iterators","http://www.brpreiss.com/books/opus4/html/page541.html","../www.brpreiss.com\books\opus4\html\page541.html","HTML","3781"," Sun, 09 Dec 2001 00:31:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Graph Traversals","http://www.brpreiss.com/books/opus4/html/page542.html","../www.brpreiss.com\books\opus4\html\page542.html","HTML","3140"," Sun, 09 Dec 2001 00:31:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementing Undirected Graphs","http://www.brpreiss.com/books/opus4/html/page543.html","../www.brpreiss.com\books\opus4\html\page543.html","HTML","2421"," Sun, 09 Dec 2001 00:31:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Using Adjacency Matrices","http://www.brpreiss.com/books/opus4/html/page544.html","../www.brpreiss.com\books\opus4\html\page544.html","HTML","4462"," Sun, 09 Dec 2001 00:31:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Using Adjacency Lists","http://www.brpreiss.com/books/opus4/html/page545.html","../www.brpreiss.com\books\opus4\html\page545.html","HTML","3859"," Sun, 09 Dec 2001 00:31:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Edge-Weighted and Vertex-Weighted Graphs","http://www.brpreiss.com/books/opus4/html/page546.html","../www.brpreiss.com\books\opus4\html\page546.html","HTML","4146"," Sun, 09 Dec 2001 00:31:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Comparison of Graph Representations","http://www.brpreiss.com/books/opus4/html/page547.html","../www.brpreiss.com\books\opus4\html\page547.html","HTML","2506"," Sun, 09 Dec 2001 00:31:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Space Comparison","http://www.brpreiss.com/books/opus4/html/page548.html","../www.brpreiss.com\books\opus4\html\page548.html","HTML","4579"," Sun, 09 Dec 2001 00:31:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Time Comparison","http://www.brpreiss.com/books/opus4/html/page549.html","../www.brpreiss.com\books\opus4\html\page549.html","HTML","7757"," Sun, 09 Dec 2001 00:31:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Graph Traversals","http://www.brpreiss.com/books/opus4/html/page550.html","../www.brpreiss.com\books\opus4\html\page550.html","HTML","3171"," Sun, 09 Dec 2001 00:31:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Depth-First Traversal","http://www.brpreiss.com/books/opus4/html/page551.html","../www.brpreiss.com\books\opus4\html\page551.html","HTML","4054"," Sun, 09 Dec 2001 00:31:31 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page552.html","../www.brpreiss.com\books\opus4\html\page552.html","HTML","3877"," Sun, 09 Dec 2001 00:31:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page553.html","../www.brpreiss.com\books\opus4\html\page553.html","HTML","3093"," Sun, 09 Dec 2001 00:31:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Breadth-First Traversal","http://www.brpreiss.com/books/opus4/html/page554.html","../www.brpreiss.com\books\opus4\html\page554.html","HTML","5311"," Sun, 09 Dec 2001 00:31:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page555.html","../www.brpreiss.com\books\opus4\html\page555.html","HTML","4385"," Sun, 09 Dec 2001 00:31:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page556.html","../www.brpreiss.com\books\opus4\html\page556.html","HTML","3004"," Sun, 09 Dec 2001 00:31:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Topological Sort","http://www.brpreiss.com/books/opus4/html/page557.html","../www.brpreiss.com\books\opus4\html\page557.html","HTML","5480"," Sun, 09 Dec 2001 00:31:38 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page558.html","../www.brpreiss.com\books\opus4\html\page558.html","HTML","5127"," Sun, 09 Dec 2001 00:31:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page559.html","../www.brpreiss.com\books\opus4\html\page559.html","HTML","2446"," Sun, 09 Dec 2001 00:31:40 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Graph Traversal Applications: Testing for Cycles and Connectedness","http://www.brpreiss.com/books/opus4/html/page560.html","../www.brpreiss.com\books\opus4\html\page560.html","HTML","2692"," Sun, 09 Dec 2001 00:31:42 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Connectedness of an Undirected Graph","http://www.brpreiss.com/books/opus4/html/page561.html","../www.brpreiss.com\books\opus4\html\page561.html","HTML","5551"," Sun, 09 Dec 2001 00:31:43 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Connectedness of a Directed Graph","http://www.brpreiss.com/books/opus4/html/page562.html","../www.brpreiss.com\books\opus4\html\page562.html","HTML","8306"," Sun, 09 Dec 2001 00:31:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Testing for Cycles in a Directed Graph","http://www.brpreiss.com/books/opus4/html/page563.html","../www.brpreiss.com\books\opus4\html\page563.html","HTML","5016"," Sun, 09 Dec 2001 00:31:46 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Shortest-Path Algorithms","http://www.brpreiss.com/books/opus4/html/page564.html","../www.brpreiss.com\books\opus4\html\page564.html","HTML","5233"," Sun, 09 Dec 2001 00:31:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Single-Source Shortest Path","http://www.brpreiss.com/books/opus4/html/page565.html","../www.brpreiss.com\books\opus4\html\page565.html","HTML","9528"," Sun, 09 Dec 2001 00:31:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Dijkstra's Algorithm","http://www.brpreiss.com/books/opus4/html/page566.html","../www.brpreiss.com\books\opus4\html\page566.html","HTML","25276"," Sun, 09 Dec 2001 00:31:54 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Data Structures for Dijkstra's Algorithm","http://www.brpreiss.com/books/opus4/html/page567.html","../www.brpreiss.com\books\opus4\html\page567.html","HTML","3676"," Sun, 09 Dec 2001 00:31:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page568.html","../www.brpreiss.com\books\opus4\html\page568.html","HTML","4675"," Sun, 09 Dec 2001 00:31:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page569.html","../www.brpreiss.com\books\opus4\html\page569.html","HTML","3575"," Sun, 09 Dec 2001 00:31:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","All-Pairs Source Shortest Path","http://www.brpreiss.com/books/opus4/html/page570.html","../www.brpreiss.com\books\opus4\html\page570.html","HTML","3492"," Sun, 09 Dec 2001 00:31:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Floyd's Algorithm","http://www.brpreiss.com/books/opus4/html/page571.html","../www.brpreiss.com\books\opus4\html\page571.html","HTML","6530"," Sun, 09 Dec 2001 00:32:00 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page572.html","../www.brpreiss.com\books\opus4\html\page572.html","HTML","4196"," Sun, 09 Dec 2001 00:32:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page573.html","../www.brpreiss.com\books\opus4\html\page573.html","HTML","3012"," Sun, 09 Dec 2001 00:32:03 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Minimum-Cost Spanning Trees","http://www.brpreiss.com/books/opus4/html/page574.html","../www.brpreiss.com\books\opus4\html\page574.html","HTML","10276"," Sun, 09 Dec 2001 00:32:04 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructing Spanning Trees","http://www.brpreiss.com/books/opus4/html/page575.html","../www.brpreiss.com\books\opus4\html\page575.html","HTML","3337"," Sun, 09 Dec 2001 00:32:05 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Minimum-Cost Spanning Trees","http://www.brpreiss.com/books/opus4/html/page576.html","../www.brpreiss.com\books\opus4\html\page576.html","HTML","3779"," Sun, 09 Dec 2001 00:32:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Prim's Algorithm","http://www.brpreiss.com/books/opus4/html/page577.html","../www.brpreiss.com\books\opus4\html\page577.html","HTML","5493"," Sun, 09 Dec 2001 00:32:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page578.html","../www.brpreiss.com\books\opus4\html\page578.html","HTML","3669"," Sun, 09 Dec 2001 00:32:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Kruskal's Algorithm","http://www.brpreiss.com/books/opus4/html/page579.html","../www.brpreiss.com\books\opus4\html\page579.html","HTML","7693"," Sun, 09 Dec 2001 00:32:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page580.html","../www.brpreiss.com\books\opus4\html\page580.html","HTML","3355"," Sun, 09 Dec 2001 00:32:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Running Time Analysis","http://www.brpreiss.com/books/opus4/html/page581.html","../www.brpreiss.com\books\opus4\html\page581.html","HTML","4729"," Sun, 09 Dec 2001 00:32:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Application: Critical Path Analysis","http://www.brpreiss.com/books/opus4/html/page582.html","../www.brpreiss.com\books\opus4\html\page582.html","HTML","12152"," Sun, 09 Dec 2001 00:32:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Implementation","http://www.brpreiss.com/books/opus4/html/page583.html","../www.brpreiss.com\books\opus4\html\page583.html","HTML","7448"," Sun, 09 Dec 2001 00:32:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exercises","http://www.brpreiss.com/books/opus4/html/page584.html","../www.brpreiss.com\books\opus4\html\page584.html","HTML","12671"," Sun, 09 Dec 2001 00:32:18 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Projects","http://www.brpreiss.com/books/opus4/html/page585.html","../www.brpreiss.com\books\opus4\html\page585.html","HTML","5282"," Sun, 09 Dec 2001 00:32:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Variables, Pointers and References","http://www.brpreiss.com/books/opus4/html/page587.html","../www.brpreiss.com\books\opus4\html\page587.html","HTML","5048"," Sun, 09 Dec 2001 00:32:22 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Pointers Are Variables","http://www.brpreiss.com/books/opus4/html/page588.html","../www.brpreiss.com\books\opus4\html\page588.html","HTML","3900"," Sun, 09 Dec 2001 01:04:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Dereferencing Pointers","http://www.brpreiss.com/books/opus4/html/page589.html","../www.brpreiss.com\books\opus4\html\page589.html","HTML","3602"," Sun, 09 Dec 2001 01:04:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","References are Not Variables","http://www.brpreiss.com/books/opus4/html/page590.html","../www.brpreiss.com\books\opus4\html\page590.html","HTML","3319"," Sun, 09 Dec 2001 01:04:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Parameter Passing","http://www.brpreiss.com/books/opus4/html/page591.html","../www.brpreiss.com\books\opus4\html\page591.html","HTML","2345"," Sun, 09 Dec 2001 01:04:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Pass By Value","http://www.brpreiss.com/books/opus4/html/page592.html","../www.brpreiss.com\books\opus4\html\page592.html","HTML","3986"," Sun, 09 Dec 2001 01:04:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Pass By Reference","http://www.brpreiss.com/books/opus4/html/page593.html","../www.brpreiss.com\books\opus4\html\page593.html","HTML","3949"," Sun, 09 Dec 2001 01:04:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Trade-off","http://www.brpreiss.com/books/opus4/html/page594.html","../www.brpreiss.com\books\opus4\html\page594.html","HTML","3090"," Sun, 09 Dec 2001 01:04:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constant Parameters","http://www.brpreiss.com/books/opus4/html/page595.html","../www.brpreiss.com\books\opus4\html\page595.html","HTML","2915"," Sun, 09 Dec 2001 01:04:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Objects and Classes","http://www.brpreiss.com/books/opus4/html/page596.html","../www.brpreiss.com\books\opus4\html\page596.html","HTML","3230"," Sun, 09 Dec 2001 01:04:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Variables and Member Functions","http://www.brpreiss.com/books/opus4/html/page597.html","../www.brpreiss.com\books\opus4\html\page597.html","HTML","3805"," Sun, 09 Dec 2001 01:04:26 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Constructors and Destructors","http://www.brpreiss.com/books/opus4/html/page598.html","../www.brpreiss.com\books\opus4\html\page598.html","HTML","3350"," Sun, 09 Dec 2001 01:04:28 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Default Constructor","http://www.brpreiss.com/books/opus4/html/page599.html","../www.brpreiss.com\books\opus4\html\page599.html","HTML","3228"," Sun, 09 Dec 2001 01:04:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Copy Constructor","http://www.brpreiss.com/books/opus4/html/page600.html","../www.brpreiss.com\books\opus4\html\page600.html","HTML","2625"," Sun, 09 Dec 2001 01:04:30 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","The Copy Constructor, Parameter Passingand Function Return Values","http://www.brpreiss.com/books/opus4/html/page601.html","../www.brpreiss.com\books\opus4\html\page601.html","HTML","3236"," Sun, 09 Dec 2001 01:04:32 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Destructors","http://www.brpreiss.com/books/opus4/html/page602.html","../www.brpreiss.com\books\opus4\html\page602.html","HTML","3265"," Sun, 09 Dec 2001 01:04:33 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Accessors and Mutators","http://www.brpreiss.com/books/opus4/html/page603.html","../www.brpreiss.com\books\opus4\html\page603.html","HTML","3960"," Sun, 09 Dec 2001 01:04:35 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Mutators","http://www.brpreiss.com/books/opus4/html/page604.html","../www.brpreiss.com\books\opus4\html\page604.html","HTML","3367"," Sun, 09 Dec 2001 01:04:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Member Access Control","http://www.brpreiss.com/books/opus4/html/page605.html","../www.brpreiss.com\books\opus4\html\page605.html","HTML","3806"," Sun, 09 Dec 2001 01:04:37 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Inheritance and Polymorphism","http://www.brpreiss.com/books/opus4/html/page606.html","../www.brpreiss.com\books\opus4\html\page606.html","HTML","2296"," Sun, 09 Dec 2001 01:04:39 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Derivation and Inheritance","http://www.brpreiss.com/books/opus4/html/page607.html","../www.brpreiss.com\books\opus4\html\page607.html","HTML","5785"," Sun, 09 Dec 2001 01:04:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Derivation and Access Control","http://www.brpreiss.com/books/opus4/html/page608.html","../www.brpreiss.com\books\opus4\html\page608.html","HTML","3539"," Sun, 09 Dec 2001 01:04:41 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Polymorphism","http://www.brpreiss.com/books/opus4/html/page609.html","../www.brpreiss.com\books\opus4\html\page609.html","HTML","8332"," Sun, 09 Dec 2001 01:04:44 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Virtual Member Functions","http://www.brpreiss.com/books/opus4/html/page610.html","../www.brpreiss.com\books\opus4\html\page610.html","HTML","3698"," Sun, 09 Dec 2001 01:04:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Abstract Classes and Concrete Classes","http://www.brpreiss.com/books/opus4/html/page611.html","../www.brpreiss.com\books\opus4\html\page611.html","HTML","4463"," Sun, 09 Dec 2001 01:04:47 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Algorithmic Abstraction","http://www.brpreiss.com/books/opus4/html/page612.html","../www.brpreiss.com\books\opus4\html\page612.html","HTML","3231"," Sun, 09 Dec 2001 01:04:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Multiple Inheritance","http://www.brpreiss.com/books/opus4/html/page613.html","../www.brpreiss.com\books\opus4\html\page613.html","HTML","4004"," Sun, 09 Dec 2001 01:04:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Run-Time Type Information and Casts","http://www.brpreiss.com/books/opus4/html/page614.html","../www.brpreiss.com\books\opus4\html\page614.html","HTML","4606"," Sun, 09 Dec 2001 01:04:52 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Templates","http://www.brpreiss.com/books/opus4/html/page615.html","../www.brpreiss.com\books\opus4\html\page615.html","HTML","4443"," Sun, 09 Dec 2001 01:04:53 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Parsed","Exceptions","http://www.brpreiss.com/books/opus4/html/page616.html","../www.brpreiss.com\books\opus4\html\page616.html","HTML","4640"," Sun, 09 Dec 2001 01:04:55 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page9.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/next_motif_gr.gif","../www.brpreiss.com\books\opus4\icons\next_motif_gr.gif","Images","172"," Sun, 09 Dec 2001 01:05:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img114.gif","../www.brpreiss.com\books\opus4\html\img114.gif","Images","62"," Sun, 09 Dec 2001 00:04:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img407.gif","../www.brpreiss.com\books\opus4\html\img407.gif","Images","63"," Sun, 09 Dec 2001 00:15:02 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img434.gif","../www.brpreiss.com\books\opus4\html\img434.gif","Images","66"," Sun, 09 Dec 2001 00:15:36 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img985.gif","../www.brpreiss.com\books\opus4\html\img985.gif","Images","58"," Sun, 09 Dec 2001 00:24:29 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","ukasiewicz, Jan","http://www.brpreiss.com/books/opus4/html/img725.gif","../www.brpreiss.com\books\opus4\html\img725.gif","Images","61"," Sun, 09 Dec 2001 00:19:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page620.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img236.gif","../www.brpreiss.com\books\opus4\html\img236.gif","Images","141"," Sun, 09 Dec 2001 00:12:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page56.html"
"Saved",", respectively,where","http://www.brpreiss.com/books/opus4/html/img237.gif","../www.brpreiss.com\books\opus4\html\img237.gif","Images","143"," Sun, 09 Dec 2001 00:12:56 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page56.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img238.gif","../www.brpreiss.com\books\opus4\html\img238.gif","Images","77"," Sun, 09 Dec 2001 00:12:57 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page56.html"
"Saved",".Then clearly algorithm","http://www.brpreiss.com/books/opus4/html/img239.gif","../www.brpreiss.com\books\opus4\html\img239.gif","Images","281"," Sun, 09 Dec 2001 00:12:58 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page56.html"
"Saved",",then algorithm","http://www.brpreiss.com/books/opus4/html/img240.gif","../www.brpreiss.com\books\opus4\html\img240.gif","Images","350"," Sun, 09 Dec 2001 00:12:59 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page56.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/cross_ref_motif.gif","../www.brpreiss.com\books\opus4\icons\cross_ref_motif.gif","Images","78"," Sun, 09 Dec 2001 01:05:25 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page166.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img1084.gif","../www.brpreiss.com\books\opus4\html\img1084.gif","Images","2471"," Sun, 09 Dec 2001 00:11:45 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page250.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img1586.gif","../www.brpreiss.com\books\opus4\html\img1586.gif","Images","81"," Sun, 09 Dec 2001 00:41:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img1587.gif","../www.brpreiss.com\books\opus4\html\img1587.gif","Images","100"," Sun, 09 Dec 2001 00:41:07 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img1588.gif","../www.brpreiss.com\books\opus4\html\img1588.gif","Images","180"," Sun, 09 Dec 2001 00:41:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","are all setsthe elements of which are drawn from","http://www.brpreiss.com/books/opus4/html/img1589.gif","../www.brpreiss.com\books\opus4\html\img1589.gif","Images","127"," Sun, 09 Dec 2001 00:41:09 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",".The set of all possible elements,","http://www.brpreiss.com/books/opus4/html/img1590.gif","../www.brpreiss.com\books\opus4\html\img1590.gif","Images","237"," Sun, 09 Dec 2001 00:41:11 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img1591.gif","../www.brpreiss.com\books\opus4\html\img1591.gif","Images","150"," Sun, 09 Dec 2001 00:41:12 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","are the same set.","http://www.brpreiss.com/books/opus4/html/img1592.gif","../www.brpreiss.com\books\opus4\html\img1592.gif","Images","150"," Sun, 09 Dec 2001 00:41:13 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",",is the set comprised of all the elements of","http://www.brpreiss.com/books/opus4/html/img1593.gif","../www.brpreiss.com\books\opus4\html\img1593.gif","Images","107"," Sun, 09 Dec 2001 00:41:14 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img1594.gif","../www.brpreiss.com\books\opus4\html\img1594.gif","Images","219"," Sun, 09 Dec 2001 00:41:15 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",", then","http://www.brpreiss.com/books/opus4/html/img1595.gif","../www.brpreiss.com\books\opus4\html\img1595.gif","Images","175"," Sun, 09 Dec 2001 00:41:16 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img1596.gif","../www.brpreiss.com\books\opus4\html\img1596.gif","Images","284"," Sun, 09 Dec 2001 00:41:17 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",".The elements of","http://www.brpreiss.com/books/opus4/html/img1597.gif","../www.brpreiss.com\books\opus4\html\img1597.gif","Images","107"," Sun, 09 Dec 2001 00:41:19 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img1598.gif","../www.brpreiss.com\books\opus4\html\img1598.gif","Images","201"," Sun, 09 Dec 2001 00:41:20 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img1599.gif","../www.brpreiss.com\books\opus4\html\img1599.gif","Images","227"," Sun, 09 Dec 2001 00:41:21 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img1600.gif","../www.brpreiss.com\books\opus4\html\img1600.gif","Images","1732"," Sun, 09 Dec 2001 00:41:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page387.html"
"Saved",",and that the running time of operator","http://www.brpreiss.com/books/opus4/html/img128.gif","../www.brpreiss.com\books\opus4\html\img128.gif","Images","83"," Sun, 09 Dec 2001 00:04:23 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page416.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img129.gif","../www.brpreiss.com\books\opus4\html\img129.gif","Images","111"," Sun, 09 Dec 2001 00:04:24 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page416.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img2281.gif","../www.brpreiss.com\books\opus4\html\img2281.gif","Images","7071"," Sun, 09 Dec 2001 00:56:06 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page523.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img2611.gif","../www.brpreiss.com\books\opus4\html\img2611.gif","Images","847"," Sun, 09 Dec 2001 01:04:08 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page617.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img2612.gif","../www.brpreiss.com\books\opus4\html\img2612.gif","Images","9585"," Sun, 09 Dec 2001 01:04:10 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/html/page617.html"
"Saved","Department of Electrical and Computer Engineering","http://www.brpreiss.com/books/opus4/icons/mail.xbm","../www.brpreiss.com\books\opus4\icons\mail.xbm","Images","506"," Sun, 09 Dec 2001 01:05:51 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/signature.html"
"Saved","Email:","http://www.brpreiss.com/books/opus4/icons/network.xbm","../www.brpreiss.com\books\opus4\icons\network.xbm","Images","515"," Sun, 09 Dec 2001 01:05:48 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/signature.html"
"Saved","URL:","http://www.brpreiss.com/books/opus4/icons/home.xbm","../www.brpreiss.com\books\opus4\icons\home.xbm","Images","506"," Sun, 09 Dec 2001 01:05:49 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/signature.html"
"Parsed","Bruno's Home Page","http://www.brpreiss.com/","../www.brpreiss.com\www.brpreiss.html","HTML","1370"," Sun, 06 Jan 2002 01:47:50 GMT","200 OK","2","http://www.brpreiss.com/books/opus4/signature.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img5.gif","../www.brpreiss.com\books\opus4\html\img5.gif","Images","636"," Sun, 09 Dec 2001 00:06:12 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page8.html"
"Saved",",this chapter also covers other asymptotic notations(","http://www.brpreiss.com/books/opus4/html/img1.gif","../www.brpreiss.com\books\opus4\html\img1.gif","Images","117"," Sun, 09 Dec 2001 00:04:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page6.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img2.gif","../www.brpreiss.com\books\opus4\html\img2.gif","Images","114"," Sun, 09 Dec 2001 00:05:18 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page6.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img3.gif","../www.brpreiss.com\books\opus4\html\img3.gif","Images","117"," Sun, 09 Dec 2001 00:05:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page6.html"
"Saved",")and develops the asymptotic properties of polynomials and logarithms.","http://www.brpreiss.com/books/opus4/html/img4.gif","../www.brpreiss.com\books\opus4\html\img4.gif","Images","102"," Sun, 09 Dec 2001 00:06:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page6.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img6.gif","../www.brpreiss.com\books\opus4\html\img6.gif","Images","3500"," Sun, 09 Dec 2001 00:06:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page35.html"
"Saved",",and the time required to store an integer result in memory is a constant,","http://www.brpreiss.com/books/opus4/html/img7.gif","../www.brpreiss.com\books\opus4\html\img7.gif","Images","105"," Sun, 09 Dec 2001 00:06:47 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img8.gif","../www.brpreiss.com\books\opus4\html\img8.gif","Images","98"," Sun, 09 Dec 2001 00:06:55 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",".I.e., the time taken to fetch the value of variable","http://www.brpreiss.com/books/opus4/html/img9.gif","../www.brpreiss.com\books\opus4\html\img9.gif","Images","183"," Sun, 09 Dec 2001 00:07:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img10.gif","../www.brpreiss.com\books\opus4\html\img10.gif","Images","73"," Sun, 09 Dec 2001 00:04:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img11.gif","../www.brpreiss.com\books\opus4\html\img11.gif","Images","64"," Sun, 09 Dec 2001 00:04:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img12.gif","../www.brpreiss.com\books\opus4\html\img12.gif","Images","73"," Sun, 09 Dec 2001 00:04:19 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",", and","http://www.brpreiss.com/books/opus4/html/img13.gif","../www.brpreiss.com\books\opus4\html\img13.gif","Images","72"," Sun, 09 Dec 2001 00:04:24 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",", respectively.","http://www.brpreiss.com/books/opus4/html/img14.gif","../www.brpreiss.com\books\opus4\html\img14.gif","Images","72"," Sun, 09 Dec 2001 00:04:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",".This is because we need to fetch two operands,","http://www.brpreiss.com/books/opus4/html/img15.gif","../www.brpreiss.com\books\opus4\html\img15.gif","Images","252"," Sun, 09 Dec 2001 00:04:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",",and the time required to return from a function is a constant,","http://www.brpreiss.com/books/opus4/html/img16.gif","../www.brpreiss.com\books\opus4\html\img16.gif","Images","90"," Sun, 09 Dec 2001 00:04:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img17.gif","../www.brpreiss.com\books\opus4\html\img17.gif","Images","109"," Sun, 09 Dec 2001 00:04:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved",",where","http://www.brpreiss.com/books/opus4/html/img18.gif","../www.brpreiss.com\books\opus4\html\img18.gif","Images","341"," Sun, 09 Dec 2001 00:05:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved","is the running time of function","http://www.brpreiss.com/books/opus4/html/img19.gif","../www.brpreiss.com\books\opus4\html\img19.gif","Images","117"," Sun, 09 Dec 2001 00:05:10 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page36.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img20.gif","../www.brpreiss.com\books\opus4\html\img20.gif","Images","283"," Sun, 09 Dec 2001 00:05:18 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img22.gif","../www.brpreiss.com\books\opus4\html\img22.gif","Images","1825"," Sun, 09 Dec 2001 00:05:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img21.gif","../www.brpreiss.com\books\opus4\html\img21.gif","Images","143"," Sun, 09 Dec 2001 00:05:26 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img23.gif","../www.brpreiss.com\books\opus4\html\img23.gif","Images","314"," Sun, 09 Dec 2001 00:05:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img24.gif","../www.brpreiss.com\books\opus4\html\img24.gif","Images","324"," Sun, 09 Dec 2001 00:05:48 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img25.gif","../www.brpreiss.com\books\opus4\html\img25.gif","Images","191"," Sun, 09 Dec 2001 00:05:49 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img26.gif","../www.brpreiss.com\books\opus4\html\img26.gif","Images","404"," Sun, 09 Dec 2001 00:05:50 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img27.gif","../www.brpreiss.com\books\opus4\html\img27.gif","Images","399"," Sun, 09 Dec 2001 00:05:56 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img28.gif","../www.brpreiss.com\books\opus4\html\img28.gif","Images","372"," Sun, 09 Dec 2001 00:13:28 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img29.gif","../www.brpreiss.com\books\opus4\html\img29.gif","Images","377"," Sun, 09 Dec 2001 00:05:56 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img30.gif","../www.brpreiss.com\books\opus4\html\img30.gif","Images","369"," Sun, 09 Dec 2001 00:05:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page37.html"
"Saved",".This time does not include the time to compute the subscript expression,nor does it include the time to access (i.e., fetch or store)the array element.","http://www.brpreiss.com/books/opus4/html/img31.gif","../www.brpreiss.com\books\opus4\html\img31.gif","Images","85"," Sun, 09 Dec 2001 00:05:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page38.html"
"Saved",".Three operand fetches are required:the first to fetch","http://www.brpreiss.com/books/opus4/html/img32.gif","../www.brpreiss.com\books\opus4\html\img32.gif","Images","261"," Sun, 09 Dec 2001 00:05:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page38.html"
"Saved",",and a value","http://www.brpreiss.com/books/opus4/html/img33.gif","../www.brpreiss.com\books\opus4\html\img33.gif","Images","157"," Sun, 09 Dec 2001 00:05:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img34.gif","../www.brpreiss.com\books\opus4\html\img34.gif","Images","328"," Sun, 09 Dec 2001 00:06:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img36.gif","../www.brpreiss.com\books\opus4\html\img36.gif","Images","1885"," Sun, 09 Dec 2001 00:06:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","using Horner's rule","http://www.brpreiss.com/books/opus4/html/img35.gif","../www.brpreiss.com\books\opus4\html\img35.gif","Images","185"," Sun, 09 Dec 2001 00:06:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img37.gif","../www.brpreiss.com\books\opus4\html\img37.gif","Images","236"," Sun, 09 Dec 2001 00:06:02 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img38.gif","../www.brpreiss.com\books\opus4\html\img38.gif","Images","312"," Sun, 09 Dec 2001 00:06:03 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img39.gif","../www.brpreiss.com\books\opus4\html\img39.gif","Images","417"," Sun, 09 Dec 2001 00:06:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img40.gif","../www.brpreiss.com\books\opus4\html\img40.gif","Images","518"," Sun, 09 Dec 2001 00:06:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img41.gif","../www.brpreiss.com\books\opus4\html\img41.gif","Images","492"," Sun, 09 Dec 2001 00:06:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img42.gif","../www.brpreiss.com\books\opus4\html\img42.gif","Images","375"," Sun, 09 Dec 2001 00:15:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img43.gif","../www.brpreiss.com\books\opus4\html\img43.gif","Images","464"," Sun, 09 Dec 2001 00:06:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img44.gif","../www.brpreiss.com\books\opus4\html\img44.gif","Images","477"," Sun, 09 Dec 2001 00:06:08 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page39.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img45.gif","../www.brpreiss.com\books\opus4\html\img45.gif","Images","636"," Sun, 09 Dec 2001 00:15:51 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img46.gif","../www.brpreiss.com\books\opus4\html\img46.gif","Images","543"," Sun, 09 Dec 2001 00:06:08 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img47.gif","../www.brpreiss.com\books\opus4\html\img47.gif","Images","1594"," Sun, 09 Dec 2001 00:06:09 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img48.gif","../www.brpreiss.com\books\opus4\html\img48.gif","Images","180"," Sun, 09 Dec 2001 00:06:10 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img49.gif","../www.brpreiss.com\books\opus4\html\img49.gif","Images","282"," Sun, 09 Dec 2001 00:06:11 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img50.gif","../www.brpreiss.com\books\opus4\html\img50.gif","Images","322"," Sun, 09 Dec 2001 00:06:12 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved",", of","http://www.brpreiss.com/books/opus4/html/img51.gif","../www.brpreiss.com\books\opus4\html\img51.gif","Images","115"," Sun, 09 Dec 2001 00:06:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img52.gif","../www.brpreiss.com\books\opus4\html\img52.gif","Images","731"," Sun, 09 Dec 2001 00:16:53 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img53.gif","../www.brpreiss.com\books\opus4\html\img53.gif","Images","300"," Sun, 09 Dec 2001 00:06:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved",".This kind of equation is called a","http://www.brpreiss.com/books/opus4/html/img54.gif","../www.brpreiss.com\books\opus4\html\img54.gif","Images","489"," Sun, 09 Dec 2001 00:06:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page40.html"
"Saved",",then we may also write","http://www.brpreiss.com/books/opus4/html/img55.gif","../www.brpreiss.com\books\opus4\html\img55.gif","Images","289"," Sun, 09 Dec 2001 00:06:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",", provided","http://www.brpreiss.com/books/opus4/html/img56.gif","../www.brpreiss.com\books\opus4\html\img56.gif","Images","319"," Sun, 09 Dec 2001 00:06:16 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img57.gif","../www.brpreiss.com\books\opus4\html\img57.gif","Images","1412"," Sun, 09 Dec 2001 00:17:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img58.gif","../www.brpreiss.com\books\opus4\html\img58.gif","Images","349"," Sun, 09 Dec 2001 00:06:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",".Of course, if we have doubts about our intuition,we can always check our result by induction:","http://www.brpreiss.com/books/opus4/html/img59.gif","../www.brpreiss.com\books\opus4\html\img59.gif","Images","163"," Sun, 09 Dec 2001 00:06:20 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img60.gif","../www.brpreiss.com\books\opus4\html\img60.gif","Images","467"," Sun, 09 Dec 2001 00:06:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img61.gif","../www.brpreiss.com\books\opus4\html\img61.gif","Images","318"," Sun, 09 Dec 2001 00:06:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",".By this assumption","http://www.brpreiss.com/books/opus4/html/img62.gif","../www.brpreiss.com\books\opus4\html\img62.gif","Images","175"," Sun, 09 Dec 2001 00:06:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img63.gif","../www.brpreiss.com\books\opus4\html\img63.gif","Images","427"," Sun, 09 Dec 2001 00:18:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img64.gif","../www.brpreiss.com\books\opus4\html\img64.gif","Images","455"," Sun, 09 Dec 2001 00:18:26 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",".Substituting Equation&nbsp;","http://www.brpreiss.com/books/opus4/html/img65.gif","../www.brpreiss.com\books\opus4\html\img65.gif","Images","114"," Sun, 09 Dec 2001 00:06:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img66.gif","../www.brpreiss.com\books\opus4\html\img66.gif","Images","759"," Sun, 09 Dec 2001 00:18:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img67.gif","../www.brpreiss.com\books\opus4\html\img67.gif","Images","169"," Sun, 09 Dec 2001 00:06:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img68.gif","../www.brpreiss.com\books\opus4\html\img68.gif","Images","812"," Sun, 09 Dec 2001 00:19:09 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page41.html"
"Saved",",we wish to find","http://www.brpreiss.com/books/opus4/html/img69.gif","../www.brpreiss.com\books\opus4\html\img69.gif","Images","170"," Sun, 09 Dec 2001 00:06:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img70.gif","../www.brpreiss.com\books\opus4\html\img70.gif","Images","248"," Sun, 09 Dec 2001 00:06:48 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img72.gif","../www.brpreiss.com\books\opus4\html\img72.gif","Images","2362"," Sun, 09 Dec 2001 00:06:49 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img71.gif","../www.brpreiss.com\books\opus4\html\img71.gif","Images","179"," Sun, 09 Dec 2001 00:06:48 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img73.gif","../www.brpreiss.com\books\opus4\html\img73.gif","Images","249"," Sun, 09 Dec 2001 00:06:50 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img74.gif","../www.brpreiss.com\books\opus4\html\img74.gif","Images","382"," Sun, 09 Dec 2001 00:06:51 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img75.gif","../www.brpreiss.com\books\opus4\html\img75.gif","Images","364"," Sun, 09 Dec 2001 00:06:51 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img76.gif","../www.brpreiss.com\books\opus4\html\img76.gif","Images","335"," Sun, 09 Dec 2001 00:06:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","iteration of the looponly if","http://www.brpreiss.com/books/opus4/html/img77.gif","../www.brpreiss.com\books\opus4\html\img77.gif","Images","77"," Sun, 09 Dec 2001 00:06:53 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","satisfies the following","http://www.brpreiss.com/books/opus4/html/img78.gif","../www.brpreiss.com\books\opus4\html\img78.gif","Images","69"," Sun, 09 Dec 2001 00:06:54 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img79.gif","../www.brpreiss.com\books\opus4\html\img79.gif","Images","436"," Sun, 09 Dec 2001 00:06:54 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img80.gif","../www.brpreiss.com\books\opus4\html\img80.gif","Images","799"," Sun, 09 Dec 2001 00:06:56 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img81.gif","../www.brpreiss.com\books\opus4\html\img81.gif","Images","1000"," Sun, 09 Dec 2001 00:21:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved",",we actually have to solve the original problem!","http://www.brpreiss.com/books/opus4/html/img82.gif","../www.brpreiss.com\books\opus4\html\img82.gif","Images","268"," Sun, 09 Dec 2001 00:06:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page42.html"
"Saved","which gives the running time","http://www.brpreiss.com/books/opus4/html/img83.gif","../www.brpreiss.com\books\opus4\html\img83.gif","Images","198"," Sun, 09 Dec 2001 00:06:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img84.gif","../www.brpreiss.com\books\opus4\html\img84.gif","Images","696"," Sun, 09 Dec 2001 00:21:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","is the probability that line&nbsp;6 of the program is executed.The probability","http://www.brpreiss.com/books/opus4/html/img85.gif","../www.brpreiss.com\books\opus4\html\img85.gif","Images","71"," Sun, 09 Dec 2001 00:06:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img86.gif","../www.brpreiss.com\books\opus4\html\img86.gif","Images","522"," Sun, 09 Dec 2001 00:06:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img87.gif","../www.brpreiss.com\books\opus4\html\img87.gif","Images","166"," Sun, 09 Dec 2001 00:06:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved",".Conversely, if we know that the array is ordered from largest to smallest,then we know that","http://www.brpreiss.com/books/opus4/html/img88.gif","../www.brpreiss.com\books\opus4\html\img88.gif","Images","114"," Sun, 09 Dec 2001 00:07:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img89.gif","../www.brpreiss.com\books\opus4\html\img89.gif","Images","122"," Sun, 09 Dec 2001 00:07:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved",".All things being equal,we can say that this will happen with probability 1/(","http://www.brpreiss.com/books/opus4/html/img90.gif","../www.brpreiss.com\books\opus4\html\img90.gif","Images","153"," Sun, 09 Dec 2001 00:22:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img91.gif","../www.brpreiss.com\books\opus4\html\img91.gif","Images","856"," Sun, 09 Dec 2001 00:23:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img92.gif","../www.brpreiss.com\books\opus4\html\img92.gif","Images","1856"," Sun, 09 Dec 2001 00:07:02 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved",",is the","http://www.brpreiss.com/books/opus4/html/img93.gif","../www.brpreiss.com\books\opus4\html\img93.gif","Images","213"," Sun, 09 Dec 2001 00:07:03 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img94.gif","../www.brpreiss.com\books\opus4\html\img94.gif","Images","83"," Sun, 09 Dec 2001 00:07:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page43.html"
"Saved","iscalled the","http://www.brpreiss.com/books/opus4/html/img95.gif","../www.brpreiss.com\books\opus4\html\img95.gif","Images","188"," Sun, 09 Dec 2001 00:07:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img96.gif","../www.brpreiss.com\books\opus4\html\img96.gif","Images","349"," Sun, 09 Dec 2001 00:07:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img97.gif","../www.brpreiss.com\books\opus4\html\img97.gif","Images","91"," Sun, 09 Dec 2001 00:07:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",", ...As it turns out,harmonic numbers often creep into the analysis of algorithms.Therefore, we should understand a little bit about how they behave.","http://www.brpreiss.com/books/opus4/html/img98.gif","../www.brpreiss.com\books\opus4\html\img98.gif","Images","90"," Sun, 09 Dec 2001 00:07:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","),","http://www.brpreiss.com/books/opus4/html/img99.gif","../www.brpreiss.com\books\opus4\html\img99.gif","Images","212"," Sun, 09 Dec 2001 00:07:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","does not exist.In other words, the summation","http://www.brpreiss.com/books/opus4/html/img100.gif","../www.brpreiss.com\books\opus4\html\img100.gif","Images","185"," Sun, 09 Dec 2001 00:04:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","goes off to infinity, but just barely.","http://www.brpreiss.com/books/opus4/html/img101.gif","../www.brpreiss.com\books\opus4\html\img101.gif","Images","154"," Sun, 09 Dec 2001 00:04:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img102.gif","../www.brpreiss.com\books\opus4\html\img102.gif","Images","185"," Sun, 09 Dec 2001 00:04:08 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Parsed","Footnotes","http://www.brpreiss.com/books/opus4/html/footnode.html","../www.brpreiss.com\books\opus4\html\footnode.html","HTML","13941"," Sun, 09 Dec 2001 00:02:34 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/icons/foot_motif.gif","../www.brpreiss.com\books\opus4\icons\foot_motif.gif","Images","87"," Sun, 09 Dec 2001 01:05:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img107.gif","../www.brpreiss.com\books\opus4\html\img107.gif","Images","215"," Sun, 09 Dec 2001 00:04:11 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img108.gif","../www.brpreiss.com\books\opus4\html\img108.gif","Images","150"," Sun, 09 Dec 2001 00:04:12 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img109.gif","../www.brpreiss.com\books\opus4\html\img109.gif","Images","2249"," Sun, 09 Dec 2001 00:04:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img110.gif","../www.brpreiss.com\books\opus4\html\img110.gif","Images","696"," Sun, 09 Dec 2001 00:12:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img111.gif","../www.brpreiss.com\books\opus4\html\img111.gif","Images","630"," Sun, 09 Dec 2001 00:12:09 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","is approximately","http://www.brpreiss.com/books/opus4/html/img112.gif","../www.brpreiss.com\books\opus4\html\img112.gif","Images","112"," Sun, 09 Dec 2001 00:04:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img113.gif","../www.brpreiss.com\books\opus4\html\img113.gif","Images","86"," Sun, 09 Dec 2001 00:04:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img115.gif","../www.brpreiss.com\books\opus4\html\img115.gif","Images","2247"," Sun, 09 Dec 2001 00:12:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img116.gif","../www.brpreiss.com\books\opus4\html\img116.gif","Images","2859"," Sun, 09 Dec 2001 00:04:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","harmonic number:","http://www.brpreiss.com/books/opus4/html/img117.gif","../www.brpreiss.com\books\opus4\html\img117.gif","Images","165"," Sun, 09 Dec 2001 00:04:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img118.gif","../www.brpreiss.com\books\opus4\html\img118.gif","Images","378"," Sun, 09 Dec 2001 00:12:41 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","is the error introduced by the fact that","http://www.brpreiss.com/books/opus4/html/img119.gif","../www.brpreiss.com\books\opus4\html\img119.gif","Images","70"," Sun, 09 Dec 2001 00:04:18 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",", but we only need the difference on the interval [1,","http://www.brpreiss.com/books/opus4/html/img120.gif","../www.brpreiss.com\books\opus4\html\img120.gif","Images","151"," Sun, 09 Dec 2001 00:04:19 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img121.gif","../www.brpreiss.com\books\opus4\html\img121.gif","Images","195"," Sun, 09 Dec 2001 00:04:20 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img123.gif","../www.brpreiss.com\books\opus4\html\img123.gif","Images","246"," Sun, 09 Dec 2001 00:04:21 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img124.gif","../www.brpreiss.com\books\opus4\html\img124.gif","Images","1153"," Sun, 09 Dec 2001 00:12:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page44.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img125.gif","../www.brpreiss.com\books\opus4\html\img125.gif","Images","1616"," Sun, 09 Dec 2001 00:12:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page45.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img126.gif","../www.brpreiss.com\books\opus4\html\img126.gif","Images","876"," Sun, 09 Dec 2001 00:12:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page45.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img127.gif","../www.brpreiss.com\books\opus4\html\img127.gif","Images","1839"," Sun, 09 Dec 2001 00:04:22 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page45.html"
"Saved",".Similarly,the running time of","http://www.brpreiss.com/books/opus4/html/img130.gif","../www.brpreiss.com\books\opus4\html\img130.gif","Images","163"," Sun, 09 Dec 2001 00:04:25 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page46.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img131.gif","../www.brpreiss.com\books\opus4\html\img131.gif","Images","194"," Sun, 09 Dec 2001 00:04:26 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page46.html"
"Saved",", where","http://www.brpreiss.com/books/opus4/html/img132.gif","../www.brpreiss.com\books\opus4\html\img132.gif","Images","217"," Sun, 09 Dec 2001 00:04:26 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page47.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img133.gif","../www.brpreiss.com\books\opus4\html\img133.gif","Images","171"," Sun, 09 Dec 2001 00:04:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page47.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img134.gif","../www.brpreiss.com\books\opus4\html\img134.gif","Images","163"," Sun, 09 Dec 2001 00:04:28 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page47.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img135.gif","../www.brpreiss.com\books\opus4\html\img135.gif","Images","304"," Sun, 09 Dec 2001 00:04:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img137.gif","../www.brpreiss.com\books\opus4\html\img137.gif","Images","2641"," Sun, 09 Dec 2001 00:04:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img136.gif","../www.brpreiss.com\books\opus4\html\img136.gif","Images","155"," Sun, 09 Dec 2001 00:04:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img138.gif","../www.brpreiss.com\books\opus4\html\img138.gif","Images","235"," Sun, 09 Dec 2001 00:04:31 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img139.gif","../www.brpreiss.com\books\opus4\html\img139.gif","Images","170"," Sun, 09 Dec 2001 00:04:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img140.gif","../www.brpreiss.com\books\opus4\html\img140.gif","Images","261"," Sun, 09 Dec 2001 00:04:33 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img141.gif","../www.brpreiss.com\books\opus4\html\img141.gif","Images","217"," Sun, 09 Dec 2001 00:04:34 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved",".Both of these are","http://www.brpreiss.com/books/opus4/html/img142.gif","../www.brpreiss.com\books\opus4\html\img142.gif","Images","146"," Sun, 09 Dec 2001 00:04:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved","is","http://www.brpreiss.com/books/opus4/html/img143.gif","../www.brpreiss.com\books\opus4\html\img143.gif","Images","145"," Sun, 09 Dec 2001 00:04:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page48.html"
"Saved",",is an","http://www.brpreiss.com/books/opus4/html/img144.gif","../www.brpreiss.com\books\opus4\html\img144.gif","Images","170"," Sun, 09 Dec 2001 00:04:36 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img145.gif","../www.brpreiss.com\books\opus4\html\img145.gif","Images","329"," Sun, 09 Dec 2001 00:04:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img146.gif","../www.brpreiss.com\books\opus4\html\img146.gif","Images","1688"," Sun, 09 Dec 2001 00:12:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img147.gif","../www.brpreiss.com\books\opus4\html\img147.gif","Images","1181"," Sun, 09 Dec 2001 00:12:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved",".and suppose","http://www.brpreiss.com/books/opus4/html/img148.gif","../www.brpreiss.com\books\opus4\html\img148.gif","Images","196"," Sun, 09 Dec 2001 00:04:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img149.gif","../www.brpreiss.com\books\opus4\html\img149.gif","Images","223"," Sun, 09 Dec 2001 00:04:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved",",where","http://www.brpreiss.com/books/opus4/html/img150.gif","../www.brpreiss.com\books\opus4\html\img150.gif","Images","195"," Sun, 09 Dec 2001 00:04:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved","to get","http://www.brpreiss.com/books/opus4/html/img151.gif","../www.brpreiss.com\books\opus4\html\img151.gif","Images","103"," Sun, 09 Dec 2001 00:04:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img152.gif","../www.brpreiss.com\books\opus4\html\img152.gif","Images","364"," Sun, 09 Dec 2001 00:04:41 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page49.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img153.gif","../www.brpreiss.com\books\opus4\html\img153.gif","Images","1842"," Sun, 09 Dec 2001 00:04:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page50.html"
"Saved",".Hence, according to our simplified model of the computer,Program&nbsp;","http://www.brpreiss.com/books/opus4/html/img154.gif","../www.brpreiss.com\books\opus4\html\img154.gif","Images","420"," Sun, 09 Dec 2001 00:04:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page50.html"
"Saved",",is a","http://www.brpreiss.com/books/opus4/html/img155.gif","../www.brpreiss.com\books\opus4\html\img155.gif","Images","191"," Sun, 09 Dec 2001 00:04:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page51.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img156.gif","../www.brpreiss.com\books\opus4\html\img156.gif","Images","342"," Sun, 09 Dec 2001 00:04:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page51.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img157.gif","../www.brpreiss.com\books\opus4\html\img157.gif","Images","1450"," Sun, 09 Dec 2001 00:12:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page51.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img158.gif","../www.brpreiss.com\books\opus4\html\img158.gif","Images","564"," Sun, 09 Dec 2001 00:12:47 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page51.html"
"Saved",".A na&#305;ve way to calculate","http://www.brpreiss.com/books/opus4/html/img159.gif","../www.brpreiss.com\books\opus4\html\img159.gif","Images","68"," Sun, 09 Dec 2001 00:04:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img160.gif","../www.brpreiss.com\books\opus4\html\img160.gif","Images","1073"," Sun, 09 Dec 2001 00:12:48 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","as follows","http://www.brpreiss.com/books/opus4/html/img161.gif","../www.brpreiss.com\books\opus4\html\img161.gif","Images","81"," Sun, 09 Dec 2001 00:04:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img162.gif","../www.brpreiss.com\books\opus4\html\img162.gif","Images","630"," Sun, 09 Dec 2001 00:04:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","as follows","http://www.brpreiss.com/books/opus4/html/img163.gif","../www.brpreiss.com\books\opus4\html\img163.gif","Images","79"," Sun, 09 Dec 2001 00:04:53 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img164.gif","../www.brpreiss.com\books\opus4\html\img164.gif","Images","586"," Sun, 09 Dec 2001 00:04:54 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img165.gif","../www.brpreiss.com\books\opus4\html\img165.gif","Images","2235"," Sun, 09 Dec 2001 00:04:55 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img166.gif","../www.brpreiss.com\books\opus4\html\img166.gif","Images","254"," Sun, 09 Dec 2001 00:04:55 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img167.gif","../www.brpreiss.com\books\opus4\html\img167.gif","Images","252"," Sun, 09 Dec 2001 00:04:56 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img168.gif","../www.brpreiss.com\books\opus4\html\img168.gif","Images","256"," Sun, 09 Dec 2001 00:04:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img169.gif","../www.brpreiss.com\books\opus4\html\img169.gif","Images","258"," Sun, 09 Dec 2001 00:04:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img170.gif","../www.brpreiss.com\books\opus4\html\img170.gif","Images","1226"," Sun, 09 Dec 2001 00:12:49 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","for some","http://www.brpreiss.com/books/opus4/html/img171.gif","../www.brpreiss.com\books\opus4\html\img171.gif","Images","107"," Sun, 09 Dec 2001 00:04:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img172.gif","../www.brpreiss.com\books\opus4\html\img172.gif","Images","292"," Sun, 09 Dec 2001 00:05:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img173.gif","../www.brpreiss.com\books\opus4\html\img173.gif","Images","428"," Sun, 09 Dec 2001 00:05:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img174.gif","../www.brpreiss.com\books\opus4\html\img174.gif","Images","1285"," Sun, 09 Dec 2001 00:12:50 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img175.gif","../www.brpreiss.com\books\opus4\html\img175.gif","Images","1022"," Sun, 09 Dec 2001 00:12:51 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".In this case, running time of Program&nbsp;","http://www.brpreiss.com/books/opus4/html/img176.gif","../www.brpreiss.com\books\opus4\html\img176.gif","Images","165"," Sun, 09 Dec 2001 00:05:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img177.gif","../www.brpreiss.com\books\opus4\html\img177.gif","Images","313"," Sun, 09 Dec 2001 00:05:02 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","for some value of","http://www.brpreiss.com/books/opus4/html/img178.gif","../www.brpreiss.com\books\opus4\html\img178.gif","Images","151"," Sun, 09 Dec 2001 00:05:03 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","which is a power of two and even.Now consider","http://www.brpreiss.com/books/opus4/html/img179.gif","../www.brpreiss.com\books\opus4\html\img179.gif","Images","73"," Sun, 09 Dec 2001 00:05:03 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",":","http://www.brpreiss.com/books/opus4/html/img180.gif","../www.brpreiss.com\books\opus4\html\img180.gif","Images","137"," Sun, 09 Dec 2001 00:05:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img181.gif","../www.brpreiss.com\books\opus4\html\img181.gif","Images","749"," Sun, 09 Dec 2001 00:12:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",").To compute","http://www.brpreiss.com/books/opus4/html/img182.gif","../www.brpreiss.com\books\opus4\html\img182.gif","Images","112"," Sun, 09 Dec 2001 00:05:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img183.gif","../www.brpreiss.com\books\opus4\html\img183.gif","Images","82"," Sun, 09 Dec 2001 00:05:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img184.gif","../www.brpreiss.com\books\opus4\html\img184.gif","Images","72"," Sun, 09 Dec 2001 00:05:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img185.gif","../www.brpreiss.com\books\opus4\html\img185.gif","Images","71"," Sun, 09 Dec 2001 00:05:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",", and finally,","http://www.brpreiss.com/books/opus4/html/img186.gif","../www.brpreiss.com\books\opus4\html\img186.gif","Images","70"," Sun, 09 Dec 2001 00:05:08 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","--all but the last of which are odd powers of","http://www.brpreiss.com/books/opus4/html/img187.gif","../www.brpreiss.com\books\opus4\html\img187.gif","Images","73"," Sun, 09 Dec 2001 00:05:09 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img188.gif","../www.brpreiss.com\books\opus4\html\img188.gif","Images","456"," Sun, 09 Dec 2001 00:05:10 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img189.gif","../www.brpreiss.com\books\opus4\html\img189.gif","Images","1389"," Sun, 09 Dec 2001 00:12:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img190.gif","../www.brpreiss.com\books\opus4\html\img190.gif","Images","605"," Sun, 09 Dec 2001 00:12:53 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".In this case, running time of Program&nbsp;","http://www.brpreiss.com/books/opus4/html/img191.gif","../www.brpreiss.com\books\opus4\html\img191.gif","Images","234"," Sun, 09 Dec 2001 00:05:11 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img192.gif","../www.brpreiss.com\books\opus4\html\img192.gif","Images","367"," Sun, 09 Dec 2001 00:05:12 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img193.gif","../www.brpreiss.com\books\opus4\html\img193.gif","Images","209"," Sun, 09 Dec 2001 00:05:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img194.gif","../www.brpreiss.com\books\opus4\html\img194.gif","Images","98"," Sun, 09 Dec 2001 00:05:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img195.gif","../www.brpreiss.com\books\opus4\html\img195.gif","Images","124"," Sun, 09 Dec 2001 00:05:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img196.gif","../www.brpreiss.com\books\opus4\html\img196.gif","Images","126"," Sun, 09 Dec 2001 00:05:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img197.gif","../www.brpreiss.com\books\opus4\html\img197.gif","Images","151"," Sun, 09 Dec 2001 00:05:16 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img198.gif","../www.brpreiss.com\books\opus4\html\img198.gif","Images","153"," Sun, 09 Dec 2001 00:05:16 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img199.gif","../www.brpreiss.com\books\opus4\html\img199.gif","Images","154"," Sun, 09 Dec 2001 00:05:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img200.gif","../www.brpreiss.com\books\opus4\html\img200.gif","Images","156"," Sun, 09 Dec 2001 00:05:19 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img201.gif","../www.brpreiss.com\books\opus4\html\img201.gif","Images","177"," Sun, 09 Dec 2001 00:05:20 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img202.gif","../www.brpreiss.com\books\opus4\html\img202.gif","Images","524"," Sun, 09 Dec 2001 00:12:54 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page52.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img203.gif","../www.brpreiss.com\books\opus4\html\img203.gif","Images","93"," Sun, 09 Dec 2001 00:05:20 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img204.gif","../www.brpreiss.com\books\opus4\html\img204.gif","Images","1745"," Sun, 09 Dec 2001 00:05:21 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved",".Adding to this the additional work done on line&nbsp;5 givesthe average running time for Program&nbsp;","http://www.brpreiss.com/books/opus4/html/img205.gif","../www.brpreiss.com\books\opus4\html\img205.gif","Images","380"," Sun, 09 Dec 2001 00:05:22 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img206.gif","../www.brpreiss.com\books\opus4\html\img206.gif","Images","482"," Sun, 09 Dec 2001 00:05:23 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","in Figure&nbsp;","http://www.brpreiss.com/books/opus4/html/img207.gif","../www.brpreiss.com\books\opus4\html\img207.gif","Images","194"," Sun, 09 Dec 2001 00:05:24 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img208.gif","../www.brpreiss.com\books\opus4\html\img208.gif","Images","294"," Sun, 09 Dec 2001 00:05:24 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img209.gif","../www.brpreiss.com\books\opus4\html\img209.gif","Images","390"," Sun, 09 Dec 2001 00:05:25 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img210.gif","../www.brpreiss.com\books\opus4\html\img210.gif","Images","2638"," Sun, 09 Dec 2001 00:05:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page53.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img211.gif","../www.brpreiss.com\books\opus4\html\img211.gif","Images","348"," Sun, 09 Dec 2001 00:05:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img212.gif","../www.brpreiss.com\books\opus4\html\img212.gif","Images","481"," Sun, 09 Dec 2001 00:05:28 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img213.gif","../www.brpreiss.com\books\opus4\html\img213.gif","Images","394"," Sun, 09 Dec 2001 00:05:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img214.gif","../www.brpreiss.com\books\opus4\html\img214.gif","Images","184"," Sun, 09 Dec 2001 00:05:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img215.gif","../www.brpreiss.com\books\opus4\html\img215.gif","Images","291"," Sun, 09 Dec 2001 00:05:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img216.gif","../www.brpreiss.com\books\opus4\html\img216.gif","Images","299"," Sun, 09 Dec 2001 00:05:31 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img217.gif","../www.brpreiss.com\books\opus4\html\img217.gif","Images","204"," Sun, 09 Dec 2001 00:05:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved",",for","http://www.brpreiss.com/books/opus4/html/img218.gif","../www.brpreiss.com\books\opus4\html\img218.gif","Images","220"," Sun, 09 Dec 2001 00:05:33 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img219.gif","../www.brpreiss.com\books\opus4\html\img219.gif","Images","156"," Sun, 09 Dec 2001 00:05:34 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","and showthat","http://www.brpreiss.com/books/opus4/html/img220.gif","../www.brpreiss.com\books\opus4\html\img220.gif","Images","211"," Sun, 09 Dec 2001 00:05:36 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img221.gif","../www.brpreiss.com\books\opus4\html\img221.gif","Images","299"," Sun, 09 Dec 2001 00:05:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img222.gif","../www.brpreiss.com\books\opus4\html\img222.gif","Images","229"," Sun, 09 Dec 2001 00:05:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","and show thatthe difference","http://www.brpreiss.com/books/opus4/html/img223.gif","../www.brpreiss.com\books\opus4\html\img223.gif","Images","242"," Sun, 09 Dec 2001 00:05:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","is (approximately)a geometric series summation.","http://www.brpreiss.com/books/opus4/html/img224.gif","../www.brpreiss.com\books\opus4\html\img224.gif","Images","156"," Sun, 09 Dec 2001 00:05:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img225.gif","../www.brpreiss.com\books\opus4\html\img225.gif","Images","545"," Sun, 09 Dec 2001 00:05:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img226.gif","../www.brpreiss.com\books\opus4\html\img226.gif","Images","615"," Sun, 09 Dec 2001 00:05:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img227.gif","../www.brpreiss.com\books\opus4\html\img227.gif","Images","561"," Sun, 09 Dec 2001 00:05:41 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img228.gif","../www.brpreiss.com\books\opus4\html\img228.gif","Images","566"," Sun, 09 Dec 2001 00:05:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img229.gif","../www.brpreiss.com\books\opus4\html\img229.gif","Images","537"," Sun, 09 Dec 2001 00:05:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img230.gif","../www.brpreiss.com\books\opus4\html\img230.gif","Images","562"," Sun, 09 Dec 2001 00:05:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img231.gif","../www.brpreiss.com\books\opus4\html\img231.gif","Images","560"," Sun, 09 Dec 2001 00:05:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page54.html"
"Saved",",and a small positive integer","http://www.brpreiss.com/books/opus4/html/img232.gif","../www.brpreiss.com\books\opus4\html\img232.gif","Images","222"," Sun, 09 Dec 2001 00:05:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page55.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img233.gif","../www.brpreiss.com\books\opus4\html\img233.gif","Images","357"," Sun, 09 Dec 2001 00:05:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page55.html"
"Saved",".For each sequence, test the hypothesis that the probabilitythat","http://www.brpreiss.com/books/opus4/html/img234.gif","../www.brpreiss.com\books\opus4\html\img234.gif","Images","249"," Sun, 09 Dec 2001 00:05:47 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page55.html"
"Saved",".(For a good source of random numbers, see Section&nbsp;","http://www.brpreiss.com/books/opus4/html/img235.gif","../www.brpreiss.com\books\opus4\html\img235.gif","Images","216"," Sun, 09 Dec 2001 00:12:55 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page55.html"
"Saved",".We say that ``","http://www.brpreiss.com/books/opus4/html/img241.gif","../www.brpreiss.com\books\opus4\html\img241.gif","Images","119"," Sun, 09 Dec 2001 00:12:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page57.html"
"Saved",",","http://www.brpreiss.com/books/opus4/html/img242.gif","../www.brpreiss.com\books\opus4\html\img242.gif","Images","130"," Sun, 09 Dec 2001 00:13:00 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page57.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img243.gif","../www.brpreiss.com\books\opus4\html\img243.gif","Images","228"," Sun, 09 Dec 2001 00:13:01 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page57.html"
"Saved",".According to Definition&nbsp;","http://www.brpreiss.com/books/opus4/html/img244.gif","../www.brpreiss.com\books\opus4\html\img244.gif","Images","233"," Sun, 09 Dec 2001 00:13:02 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img245.gif","../www.brpreiss.com\books\opus4\html\img245.gif","Images","197"," Sun, 09 Dec 2001 00:13:03 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img246.gif","../www.brpreiss.com\books\opus4\html\img246.gif","Images","973"," Sun, 09 Dec 2001 00:13:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved",".I.e.,","http://www.brpreiss.com/books/opus4/html/img247.gif","../www.brpreiss.com\books\opus4\html\img247.gif","Images","211"," Sun, 09 Dec 2001 00:13:04 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img248.gif","../www.brpreiss.com\books\opus4\html\img248.gif","Images","137"," Sun, 09 Dec 2001 00:13:05 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved","is greater thanthe function","http://www.brpreiss.com/books/opus4/html/img249.gif","../www.brpreiss.com\books\opus4\html\img249.gif","Images","177"," Sun, 09 Dec 2001 00:13:06 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved","will do,as will","http://www.brpreiss.com/books/opus4/html/img250.gif","../www.brpreiss.com\books\opus4\html\img250.gif","Images","321"," Sun, 09 Dec 2001 00:13:07 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved",".(See Figure&nbsp;","http://www.brpreiss.com/books/opus4/html/img251.gif","../www.brpreiss.com\books\opus4\html\img251.gif","Images","289"," Sun, 09 Dec 2001 00:13:08 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img253.gif","../www.brpreiss.com\books\opus4\html\img253.gif","Images","3467"," Sun, 09 Dec 2001 00:13:10 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img252.gif","../www.brpreiss.com\books\opus4\html\img252.gif","Images","362"," Sun, 09 Dec 2001 00:13:09 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page58.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img254.gif","../www.brpreiss.com\books\opus4\html\img254.gif","Images","273"," Sun, 09 Dec 2001 00:13:11 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved",",then","http://www.brpreiss.com/books/opus4/html/img255.gif","../www.brpreiss.com\books\opus4\html\img255.gif","Images","268"," Sun, 09 Dec 2001 00:13:11 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img256.gif","../www.brpreiss.com\books\opus4\html\img256.gif","Images","231"," Sun, 09 Dec 2001 00:13:12 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img257.gif","../www.brpreiss.com\books\opus4\html\img257.gif","Images","575"," Sun, 09 Dec 2001 00:13:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img258.gif","../www.brpreiss.com\books\opus4\html\img258.gif","Images","585"," Sun, 09 Dec 2001 00:13:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img259.gif","../www.brpreiss.com\books\opus4\html\img259.gif","Images","171"," Sun, 09 Dec 2001 00:13:15 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","are both","http://www.brpreiss.com/books/opus4/html/img260.gif","../www.brpreiss.com\books\opus4\html\img260.gif","Images","187"," Sun, 09 Dec 2001 00:13:16 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved",", but they are not equal.","http://www.brpreiss.com/books/opus4/html/img261.gif","../www.brpreiss.com\books\opus4\html\img261.gif","Images","144"," Sun, 09 Dec 2001 00:13:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img262.gif","../www.brpreiss.com\books\opus4\html\img262.gif","Images","286"," Sun, 09 Dec 2001 00:13:17 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","provided that","http://www.brpreiss.com/books/opus4/html/img263.gif","../www.brpreiss.com\books\opus4\html\img263.gif","Images","281"," Sun, 09 Dec 2001 00:13:18 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","is an invertible function.However, while we may write","http://www.brpreiss.com/books/opus4/html/img264.gif","../www.brpreiss.com\books\opus4\html\img264.gif","Images","109"," Sun, 09 Dec 2001 00:13:19 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page59.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img265.gif","../www.brpreiss.com\books\opus4\html\img265.gif","Images","282"," Sun, 09 Dec 2001 00:13:20 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img266.gif","../www.brpreiss.com\books\opus4\html\img266.gif","Images","283"," Sun, 09 Dec 2001 00:13:21 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img267.gif","../www.brpreiss.com\books\opus4\html\img267.gif","Images","131"," Sun, 09 Dec 2001 00:13:21 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","?(Theorems&nbsp;","http://www.brpreiss.com/books/opus4/html/img268.gif","../www.brpreiss.com\books\opus4\html\img268.gif","Images","131"," Sun, 09 Dec 2001 00:13:22 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","related when","http://www.brpreiss.com/books/opus4/html/img269.gif","../www.brpreiss.com\books\opus4\html\img269.gif","Images","133"," Sun, 09 Dec 2001 00:13:23 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","?(Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img270.gif","../www.brpreiss.com\books\opus4\html\img270.gif","Images","230"," Sun, 09 Dec 2001 00:13:24 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img271.gif","../www.brpreiss.com\books\opus4\html\img271.gif","Images","535"," Sun, 09 Dec 2001 00:13:25 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img272.gif","../www.brpreiss.com\books\opus4\html\img272.gif","Images","76"," Sun, 09 Dec 2001 00:13:25 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and two constants","http://www.brpreiss.com/books/opus4/html/img273.gif","../www.brpreiss.com\books\opus4\html\img273.gif","Images","76"," Sun, 09 Dec 2001 00:13:26 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img274.gif","../www.brpreiss.com\books\opus4\html\img274.gif","Images","70"," Sun, 09 Dec 2001 00:13:27 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","such that","http://www.brpreiss.com/books/opus4/html/img275.gif","../www.brpreiss.com\books\opus4\html\img275.gif","Images","71"," Sun, 09 Dec 2001 00:13:28 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img276.gif","../www.brpreiss.com\books\opus4\html\img276.gif","Images","254"," Sun, 09 Dec 2001 00:08:28 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img277.gif","../www.brpreiss.com\books\opus4\html\img277.gif","Images","131"," Sun, 09 Dec 2001 00:08:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img278.gif","../www.brpreiss.com\books\opus4\html\img278.gif","Images","258"," Sun, 09 Dec 2001 00:08:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img279.gif","../www.brpreiss.com\books\opus4\html\img279.gif","Images","129"," Sun, 09 Dec 2001 00:08:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img280.gif","../www.brpreiss.com\books\opus4\html\img280.gif","Images","246"," Sun, 09 Dec 2001 00:08:31 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".Consider the sum","http://www.brpreiss.com/books/opus4/html/img281.gif","../www.brpreiss.com\books\opus4\html\img281.gif","Images","254"," Sun, 09 Dec 2001 00:08:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img282.gif","../www.brpreiss.com\books\opus4\html\img282.gif","Images","233"," Sun, 09 Dec 2001 00:08:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img283.gif","../www.brpreiss.com\books\opus4\html\img283.gif","Images","1219"," Sun, 09 Dec 2001 00:08:33 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img284.gif","../www.brpreiss.com\books\opus4\html\img284.gif","Images","530"," Sun, 09 Dec 2001 00:08:34 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img285.gif","../www.brpreiss.com\books\opus4\html\img285.gif","Images","187"," Sun, 09 Dec 2001 00:08:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",", respectively,the","http://www.brpreiss.com/books/opus4/html/img286.gif","../www.brpreiss.com\books\opus4\html\img286.gif","Images","188"," Sun, 09 Dec 2001 00:08:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".The meaning of","http://www.brpreiss.com/books/opus4/html/img287.gif","../www.brpreiss.com\books\opus4\html\img287.gif","Images","353"," Sun, 09 Dec 2001 00:08:36 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","is this context isthe","http://www.brpreiss.com/books/opus4/html/img288.gif","../www.brpreiss.com\books\opus4\html\img288.gif","Images","302"," Sun, 09 Dec 2001 00:08:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for integers all","http://www.brpreiss.com/books/opus4/html/img289.gif","../www.brpreiss.com\books\opus4\html\img289.gif","Images","377"," Sun, 09 Dec 2001 00:08:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img290.gif","../www.brpreiss.com\books\opus4\html\img290.gif","Images","169"," Sun, 09 Dec 2001 00:08:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".Then","http://www.brpreiss.com/books/opus4/html/img291.gif","../www.brpreiss.com\books\opus4\html\img291.gif","Images","322"," Sun, 09 Dec 2001 00:08:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img292.gif","../www.brpreiss.com\books\opus4\html\img292.gif","Images","1187"," Sun, 09 Dec 2001 00:08:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","required by Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img293.gif","../www.brpreiss.com\books\opus4\html\img293.gif","Images","80"," Sun, 09 Dec 2001 00:08:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","in which","http://www.brpreiss.com/books/opus4/html/img294.gif","../www.brpreiss.com\books\opus4\html\img294.gif","Images","314"," Sun, 09 Dec 2001 00:08:41 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for some limit","http://www.brpreiss.com/books/opus4/html/img295.gif","../www.brpreiss.com\books\opus4\html\img295.gif","Images","356"," Sun, 09 Dec 2001 00:08:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",",then","http://www.brpreiss.com/books/opus4/html/img296.gif","../www.brpreiss.com\books\opus4\html\img296.gif","Images","124"," Sun, 09 Dec 2001 00:08:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img297.gif","../www.brpreiss.com\books\opus4\html\img297.gif","Images","276"," Sun, 09 Dec 2001 00:08:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img298.gif","../www.brpreiss.com\books\opus4\html\img298.gif","Images","438"," Sun, 09 Dec 2001 00:08:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",",it is possible to find a value","http://www.brpreiss.com/books/opus4/html/img299.gif","../www.brpreiss.com\books\opus4\html\img299.gif","Images","52"," Sun, 09 Dec 2001 00:08:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img300.gif","../www.brpreiss.com\books\opus4\html\img300.gif","Images","473"," Sun, 09 Dec 2001 00:08:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",",then there exists a corresponding","http://www.brpreiss.com/books/opus4/html/img301.gif","../www.brpreiss.com\books\opus4\html\img301.gif","Images","71"," Sun, 09 Dec 2001 00:08:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img302.gif","../www.brpreiss.com\books\opus4\html\img302.gif","Images","1251"," Sun, 09 Dec 2001 00:08:47 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img303.gif","../www.brpreiss.com\books\opus4\html\img303.gif","Images","1266"," Sun, 09 Dec 2001 00:13:29 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".Thus,","http://www.brpreiss.com/books/opus4/html/img304.gif","../www.brpreiss.com\books\opus4\html\img304.gif","Images","267"," Sun, 09 Dec 2001 00:13:30 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","exists,then the sum","http://www.brpreiss.com/books/opus4/html/img305.gif","../www.brpreiss.com\books\opus4\html\img305.gif","Images","313"," Sun, 09 Dec 2001 00:13:31 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","which,by the transitive property (see Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img306.gif","../www.brpreiss.com\books\opus4\html\img306.gif","Images","187"," Sun, 09 Dec 2001 00:13:32 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","asymptotically approaches a constant as","http://www.brpreiss.com/books/opus4/html/img307.gif","../www.brpreiss.com\books\opus4\html\img307.gif","Images","227"," Sun, 09 Dec 2001 00:13:35 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img308.gif","../www.brpreiss.com\books\opus4\html\img308.gif","Images","186"," Sun, 09 Dec 2001 00:13:36 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img309.gif","../www.brpreiss.com\books\opus4\html\img309.gif","Images","892"," Sun, 09 Dec 2001 00:13:37 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".Thus, Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img310.gif","../www.brpreiss.com\books\opus4\html\img310.gif","Images","432"," Sun, 09 Dec 2001 00:13:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",",where","http://www.brpreiss.com/books/opus4/html/img311.gif","../www.brpreiss.com\books\opus4\html\img311.gif","Images","151"," Sun, 09 Dec 2001 00:13:38 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img312.gif","../www.brpreiss.com\books\opus4\html\img312.gif","Images","486"," Sun, 09 Dec 2001 00:13:39 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".Consider the product","http://www.brpreiss.com/books/opus4/html/img313.gif","../www.brpreiss.com\books\opus4\html\img313.gif","Images","136"," Sun, 09 Dec 2001 00:13:40 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img314.gif","../www.brpreiss.com\books\opus4\html\img314.gif","Images","233"," Sun, 09 Dec 2001 00:13:41 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img315.gif","../www.brpreiss.com\books\opus4\html\img315.gif","Images","887"," Sun, 09 Dec 2001 00:13:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img316.gif","../www.brpreiss.com\books\opus4\html\img316.gif","Images","472"," Sun, 09 Dec 2001 00:13:42 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","and","http://www.brpreiss.com/books/opus4/html/img317.gif","../www.brpreiss.com\books\opus4\html\img317.gif","Images","411"," Sun, 09 Dec 2001 00:13:43 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".By Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img318.gif","../www.brpreiss.com\books\opus4\html\img318.gif","Images","364"," Sun, 09 Dec 2001 00:13:44 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".I.e., we are able to determine the asymptotic behavior ofthe product without having to go through the gory details ofcalculating that","http://www.brpreiss.com/books/opus4/html/img319.gif","../www.brpreiss.com\books\opus4\html\img319.gif","Images","305"," Sun, 09 Dec 2001 00:13:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img320.gif","../www.brpreiss.com\books\opus4\html\img320.gif","Images","557"," Sun, 09 Dec 2001 00:13:45 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img321.gif","../www.brpreiss.com\books\opus4\html\img321.gif","Images","483"," Sun, 09 Dec 2001 00:13:46 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","such that","http://www.brpreiss.com/books/opus4/html/img322.gif","../www.brpreiss.com\books\opus4\html\img322.gif","Images","73"," Sun, 09 Dec 2001 00:13:47 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img323.gif","../www.brpreiss.com\books\opus4\html\img323.gif","Images","262"," Sun, 09 Dec 2001 00:13:48 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img324.gif","../www.brpreiss.com\books\opus4\html\img324.gif","Images","510"," Sun, 09 Dec 2001 00:13:49 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img325.gif","../www.brpreiss.com\books\opus4\html\img325.gif","Images","471"," Sun, 09 Dec 2001 00:13:50 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".","http://www.brpreiss.com/books/opus4/html/img326.gif","../www.brpreiss.com\books\opus4\html\img326.gif","Images","285"," Sun, 09 Dec 2001 00:13:51 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img327.gif","../www.brpreiss.com\books\opus4\html\img327.gif","Images","1156"," Sun, 09 Dec 2001 00:13:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img328.gif","../www.brpreiss.com\books\opus4\html\img328.gif","Images","236"," Sun, 09 Dec 2001 00:13:52 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","for","http://www.brpreiss.com/books/opus4/html/img329.gif","../www.brpreiss.com\books\opus4\html\img329.gif","Images","246"," Sun, 09 Dec 2001 00:13:53 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img330.gif","../www.brpreiss.com\books\opus4\html\img330.gif","Images","845"," Sun, 09 Dec 2001 00:13:54 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","which is clearly","http://www.brpreiss.com/books/opus4/html/img331.gif","../www.brpreiss.com\books\opus4\html\img331.gif","Images","203"," Sun, 09 Dec 2001 00:13:55 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".If we add to","http://www.brpreiss.com/books/opus4/html/img332.gif","../www.brpreiss.com\books\opus4\html\img332.gif","Images","141"," Sun, 09 Dec 2001 00:13:56 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",",then by Theorem&nbsp;","http://www.brpreiss.com/books/opus4/html/img333.gif","../www.brpreiss.com\books\opus4\html\img333.gif","Images","208"," Sun, 09 Dec 2001 00:13:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".I.e.,","http://www.brpreiss.com/books/opus4/html/img334.gif","../www.brpreiss.com\books\opus4\html\img334.gif","Images","354"," Sun, 09 Dec 2001 00:13:57 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved",".The combination of the fact that","http://www.brpreiss.com/books/opus4/html/img335.gif","../www.brpreiss.com\books\opus4\html\img335.gif","Images","383"," Sun, 09 Dec 2001 00:13:58 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","","http://www.brpreiss.com/books/opus4/html/img336.gif","../www.brpreiss.com\books\opus4\html\img336.gif","Images","242"," Sun, 09 Dec 2001 00:13:59 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page60.html"
"Saved","diverges as","http://www.brpreiss.com/books/opus4/html/img353.gif","../www.brpreiss.com\books\opus4\html\img353.gif","Images","119"," Sun, 09 Dec 2001 00:14:13 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page62.html"
"Saved","for all integers","http://www.brpreiss.com/books/opus4/html/img354.gif","../www.brpreiss.com\books\opus4\html\img354.gif","Images","162"," Sun, 09 Dec 2001 00:14:14 GMT","200 OK","3","http://www.brpreiss.com/books/opus4/html/page62.html"


WebZIP Task Summary

Last Run: |
Next Run: |
Total Files: |
Size: | bytes
Status: |
Pages
/ 		Media
/ 		Errors
/
Category:
Title:
URL:
Local URL:
Save-to Folder:

Download Method

The following file types will be downloaded:
.

Pages will be downloaded from: .
Media files will be downloaded from:
WebZIP will download pages deep from the start page.

© 2000 Spidersoft. All rights reserved.


Copyright Notice

Copyright © 1994-2002 by Bruno R. Preiss, P.Eng. All rights reserved.

Copyright of this document is owned by Bruno R. Preiss. Permission to view, to copy for the purpose of viewing, and to copy for the purpose of printing this document is hereby granted, provided that any copies made are for personal use only, and that copies are used only for non-commercial purposes, and that the above copyright notice appears in any copies made, and that both that copyright notice and this permission notice appear. All other rights reserved.

To request permission to reproduce all or part of this information for commercial purposes, please contact:

Bruno R. Preiss, Ph.D., P.Eng.
335 Calvington Place
Waterloo, ON N2T 1R1
Canada

This document is provided as is without warranty or guarantee of any kind, either express or implied, concerning the content or accuracy of the information provided. This document may include technical inaccuracies or typographical errors. Changes are periodically made to the information herein.

Bruno
Copyright © 1994-2002 by Bruno R. Preiss, P.Eng. All rights reserved.


CV and Resume

  • My Curriculum Vita is available in html and pdf.
  • My Résumé is available in html, pdf, MS Word, and plain ASCII.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


 Website of Bruno R. Preiss
next up previous contents external
Next: Contents Up: Bruno's Home Page

Website of Bruno R. Preiss

Bruno R. Preiss, B.A.Sc., M.A.Sc., Ph.D., P.Eng.

  • Contents
  • Research
    • Publications
    • Presentations
  • Opuses (Opera?)
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C++
    • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C#
  • Personal
    • Home Address
    • CV and Resume
  • References

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Bruno's Home Page

 | Website |
Research | Bruno | Opuses
 | Personal |

Copyright © 1999-2002 by Bruno R. Preiss, P.Eng. All rights reserved.


Opuses (Opera?)

  • Data Structures and Algorithms with Object-Oriented Design Patterns in C++
  • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
  • Data Structures and Algorithms with Object-Oriented Design Patterns in C#

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Contents

  • Research
    • Publications
      • Books
        • Data Structures and Algorithms with Object-Oriented Design Patterns in C#
        • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
        • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java
        • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++
        • Data Structures and Algorithms with Object-Oriented Design Patterns in C++
      • Journal Articles
        • From Design Patterns to Parallel Architecture Skeletons
        • Effects of the Checkpoint Interval on Time and Space in Time Warp
        • Optimal Memory Management for Time Warp Parallel Simulation
        • Simulating Continuous Systems with Piecewise-Linear Signals Using Time Warp
        • Short Packet Transfer Performance in Local Area Ring Networks
        • A Nonuniform Detector Aperture for CT
        • On the Transport of Charged Particles Through Spongy Materials
      • Conference Papers
        • Building Parallel Applications using Design Patterns
        • Using Object-Oriented Techniques for Realizing Parallel Architectural Skeletons
        • Issues in Joint Undergraduate Software Engineering Degree Program Design
        • Architectural Skeletons: The Re-Usable Building-Blocks for Parallel Applications
        • Design Patterns for the Data Structures and Algorithms Course
        • The Parsimony Project: A Distributed Simulation Testbed in Java
        • Memory Management Techniques for Time Warp on a Distributed Memory Machine
        • An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation
        • On the Performance of a Multi-Threaded RISC Architecture
        • Selecting the Checkpoint Interval in Time Warp Parallel Simulation
        • A Unifying Framework for Distributed Routing Algorithms
        • On the Trade-Off between Time and Space in Optimistic Parallel Discrete-Event Simulation
        • Parallel Instance Discrete-Event Simulation Using a Vector Uniprocessor
        • Multi-Threaded Pipelining in a RISC Processor
        • Null Message Cancellation in Conservative Distributed Simulation
        • Performance of Discrete Event Simulation on a Multiprocessor Using Optimistic and Conservative Synchronization
        • The Impact of Lookahead on the Performance of Conservative Distributed Simulation
        • The Role of Knowledge in Distributed Simulation
        • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
        • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
        • Semi-Static Dataflow
        • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor
        • Data Flow on a Queue Machine
        • Short-Packet Transfer Performance in Local Area Rings
      • Patents
        • Method, System and Apparatus For Provisioning Services to Subscribers in a Telecommunications Network
        • Interactive Voice Response System and Method
        • Telecommunication Architecture
        • Strategy for Negotiation of Telecommunication Resources
        • Method and System for Configuring Communication Resources
      • Technical Reports
        • Transputer System User Guide-Second Edition
        • Effects of the Checkpoint Interval on Time and Space in Time Warp
        • Transputer System User Guide
        • YADDES-Yet Another Distributed Discrete Event Simulator: User Manual
        • Prediction and Lookahead in Distributed Simulation
        • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
        • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
        • Semi-static Dataflow
        • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor
      • Theses
        • Data Flow on a Queue Machine
        • Design and Simulation of a Data-Flow Multiprocessor
        • LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks
      • Unpublished Manuscripts
        • A Template-based Model for Developing Parallel Applications on a Network Cluster
        • A Pattern-Based Model for Developing Parallel Applications Using a Network of Processors
        • A Skeleton-Based Model for Developing Parallel Applications Using a Network of Processors
        • An Attributed, Time-Delayed Rendezvous Model for Parallel Discrete Event Simulation
        • Dynamic Rescheduling of Tasks for the Macro-Dataflow Paradigm
        • Ynot Logic Simulator: A Literate C++ Program
        • A Time-Delayed-Rendezvous Model for Parallel Discrete Event Simulation
        • A New Model for Parallel Discrete Event Simulation
        • Parallel Simulation and Lotos: Research In Progress
        • Hobbes: A Multi-Threaded Superscalar Architecture
        • Dynamic Checkpoint Interval Selection in Time Warp Simulation
        • Yaddes-Yet Another Distributed Discrete Event Simulator
        • Queue Machines
        • Effective Memory Bandwidth of Band-Connected Partial Crossbars
        • An Occam Compiler for a Dataflow Multiprocessor (Extended Abstract)
    • Presentations
  • Opuses (Opera?)
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C++
    • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C#
  • Personal
    • Home Address
    • CV and Resume
  • References

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Journal Articles

  • From Design Patterns to Parallel Architecture Skeletons
  • Effects of the Checkpoint Interval on Time and Space in Time Warp
  • Optimal Memory Management for Time Warp Parallel Simulation
  • Simulating Continuous Systems with Piecewise-Linear Signals Using Time Warp
  • Short Packet Transfer Performance in Local Area Ring Networks
  • A Nonuniform Detector Aperture for CT
  • On the Transport of Charged Particles Through Spongy Materials

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


From Design Patterns to Parallel Architecture Skeletons

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. Journal of Parallel and Distributed Computing, June 2001. (To appear)[8].

The concept of design patterns has been extensively studied and applied in the context of object-oriented software design. Similar ideas are being explored in other areas of computing as well. Over the past several years, researchers have been experimenting with the feasibility of employing design-patterns related concepts in the parallel computing domain. In the past, several pattern-based systems have been developed with the intention to facilitate faster parallel application development through the use of pre-implemented and reusable components that are based on frequently used parallel computing design patterns. However, most of these systems face several serious limitations such as limited flexibility, zero extensibility, and ad hoc nature of their components. Lack of flexibility in a parallel programming system limits a programmer to using only the high-level components provided by the system. Lack of extensibility here refers to the fact that most of the existing pattern-based parallel programming systems come with a set of pre-built patterns integrated into the system. However, the system provides no obvious way of increasing the repertoire of patterns when need arises. Also, most of these systems do not offer any generic view of a parallel computing pattern, a fact which may be at the root of several of their shortcomings. This research proposes a generic (i.e., pattern- and application-independent) model for realizing and using parallel design patterns. The term ``Parallel Architectural Skeleton'' is used to represent the set of generic attributes associated with a pattern. The Parallel Architectural Skeleton Model (PASM) is based on the message-passing paradigm, which makes it suitable for a LAN of workstations and PCs. The model is flexible as it allows the intermixing of high-level patterns with low-level message-passing primitives. An object-oriented and library-based implementation of the model has been completed using C++ and MPI, without necessitating any language extension. The generic model and the library-based implementation allow new patterns to be defined and included into the system. The skeleton-library serves as a ``framework'' for the systematic, hierarchical development of network-oriented parallel applications.

Copyright © 2001 by Journal of Parallel and Distributed Computing.

external Full text. external BibTeX entry.

Website of Bruno R. Preiss
next up previous contents external
Next: Effects of the Checkpoint Up: Journal Articles Previous: Journal Articles

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Effects of the Checkpoint Interval on Time and Space in Time Warp

Bruno Richard Preiss, Wayne Mervin Loucks, and Ian Donald MacIntyre. ACM Trans. on Modeling and Computer Simulation, 4(3):223-253, July 1994. (Accepted April 1994.)[58].

Optimistically synchronized parallel discrete-event simulation is based on the use of communicating sequential processes. Optimistic synchronization means that the processes proceed under the assumption that a synchronized execution schedule is fortuitous. Periodic checkpointing of the state of a process allows the process to roll back to an earlier state when synchronization errors are detected. This paper examines the effects of varying the checkpoint interval on the execution time and memory space needed to perform a parallel simulation.

The empirical results presented in this paper were obtained from the simulation of closed stochastic queueing networks with several different topologies. Various intra-processor process scheduling algorithms and both lazy and aggressive cancellation strategies are considered. The empirical results are compared with analytical formulae predicting time-optimal checkpoint intervals. Two modes of operation, throttling and thrashing have been noted and their effect examined. As the checkpoint interval is increased from one, there is a throttling effect among processes on the same processor which improves performance. When the checkpoint interval is made too large, there is a thrashing effect caused by interaction between processes on different processors. It is shown that the time-optimal and space-optimal checkpoint intervals are not the same. Furthermore, a checkpoint interval that is too small adversely affects space more than time; whereas, a checkpoint interval that is too large adversely affects time more than space.

Copyright © 1994 by Association for Computing Machinery, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Optimal Memory Management for Time Warp Parallel Simulation

Yi-Bing Lin and Bruno Richard Preiss. ACM Trans. on Modeling and Computer Simulation, 1(4):283-307, October 1991. (Accepted May 1992. Published September 1992.)[14].

Recently there has been a great deal of interest in performance evaluation for parallel simulation. Most work is devoted to the time complexity and assumes that the amount of memory available for parallel simulation is unlimited. This paper studies the space complexity of parallel simulation. Our goal is to design an efficient memory management protocol which guarantees that the memory consumption of parallel simulation is of the same order as sequential simulation. (Such an algorithm is referred to as optimal.) We first derive the relationships among the space complexities of sequential simulation, Chandy-Misra simulation, and Time Warp simulation. We show that Chandy-Misra may consume more storage than sequential simulation, or vice versa. Then we show that Time Warp always consumes more memory than sequential simulation. Then we describe cancelback, an optimal Time Warp memory management protocol proposed by Jefferson. Although cancelback is considered as a complete solution for the storage management problem in Time Warp, some efficiency issues in implementing this algorithm must be considered. In this paper, we propose an optimal algorithm called artificial rollback. We show that this algorithm is easy to implement and analyze. An implementation of artificial rollback is given, which is integrated with processor scheduling to avoid inefficiency that may occur in cancelback.

Copyright © 1991 by Association for Computing Machinery, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Simulating Continuous Systems with Piecewise-Linear Signals Using Time Warp

Barry T. W. Kwok and Bruno Richard Preiss. International Journal in Computer Simulation, 1(1):59-79, 1991[13].

Recently, an approach using the discrete event paradigm for the simulation of continuous systems has been developed. This approach is based on the use of piecewise-linear approximations for the representation of continuous, time-varying quantities. The focus of this paper is to show how this new technique can be implemented on a multiprocessor. Our hypothesis is that the Time Warp algorithm is the natural protocol for parallel simulation using continuous, piecewise-linear signals.

We have implemented a prototype simulator and measured its performance on a multiprocessor using several benchmarks. On the basis of these benchmarks we show that the best performance is achieved when using a scheduling algorithm that gives priority to messages with lower timestamps and when the lazy-cancellation variant of Time Warp is used. In this case, the performance of our prototype simulator compares quite favourably with the estimates of optimum performance based on a critical path analysis of the sequential simulation. In fact, our results indicate that in certain cases the performance of the Time Warp algorithm using lazy cancellation can exceed the conservative optimum estimate obtained from the critical path analysis.

Copyright © 1991 by Ablex Publishing Corporation.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Short Packet Transfer Performance in Local Area Ring Networks

Wayne Mervin Loucks, V. Carl Hamacher, Bruno Richard Preiss, and Luke Wong. IEEE Trans. on Computers, C-34(11):1006-1014, November 1985[19].

The performance of various access control protocols for bit-serial local area computer network (LACN) rings is studied. Applications in which message packets are of fixed length and shorter than the total inherent propagation delay around the ring are the focus of attention. Token, slotted, and static and dynamic insertion rings are included in the study. In all cases, the transmitting station is responsible for removing its transmitted packet from the ring. Under this type of removal rule, it is possible for the stations of all types of ring structures to execute their access control algorithms with only a short fixed in-line delay in each station. The insertion rings dynamically switch longer delays (insertion registers) into the ring when they are transmitting a packet. The transmitter-remove rule operates in such a way that hogging of the ring transmission facility by a subset of stations cannot occur. Expressions that approximate average transfer time as a function of utilization are derived for all ring types and are check by simulation. The expressions are found to be quite accurate at low ring utilization levels, which is the case of most importance for LACN's; but some of them exhibit significant errors at high utilization levels. For the assumed short-packet environment, it is found that token rings exhibit the slowest transfer times, while dynamic insertion rings are fastest.

Copyright © 1985 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Nonuniform Detector Aperture for CT

R. M. Henkelman and Bruno Richard Preiss. Journal of Computer Assisted Tomography, 5(3):401-408, June 1981[9].

The finite width of the X-ray beam determined by the source and detector aperture limits the resolution of computed tomography (CT) images. Improved resolution can be obtained by reducing the width of the detector, but at a cost in dose utilization efficiency. We show that it is possible to improve the resolution of a CT image with a minimum cost in dose efficiency by varying the sensitivity (apodizing) the detector response function. Further, we demonstrate how to construct detectors with nonuniform response by changing the reflective coating on the scintillator. Finally, we show improved calculated point response functions for such modified detectors.

Copyright © 1981 by Raven Press, New York.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


On the Transport of Charged Particles Through Spongy Materials

R. M. Henkelman and Bruno Richard Preiss. PhysMed. Biol., 26(6):1151-1156, June 1981[10].

1. Introduction, 2. Calculation, 3. Results, 4. Discussion, Acknowledgements, References.

Copyright © 1981 by The Institute of Physics.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Conference Papers

  • Building Parallel Applications using Design Patterns
  • Using Object-Oriented Techniques for Realizing Parallel Architectural Skeletons
  • Issues in Joint Undergraduate Software Engineering Degree Program Design
  • Architectural Skeletons: The Re-Usable Building-Blocks for Parallel Applications
  • Design Patterns for the Data Structures and Algorithms Course
  • The Parsimony Project: A Distributed Simulation Testbed in Java
  • Memory Management Techniques for Time Warp on a Distributed Memory Machine
  • An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation
  • On the Performance of a Multi-Threaded RISC Architecture
  • Selecting the Checkpoint Interval in Time Warp Parallel Simulation
  • A Unifying Framework for Distributed Routing Algorithms
  • On the Trade-Off between Time and Space in Optimistic Parallel Discrete-Event Simulation
  • Parallel Instance Discrete-Event Simulation Using a Vector Uniprocessor
  • Multi-Threaded Pipelining in a RISC Processor
  • Null Message Cancellation in Conservative Distributed Simulation
  • Performance of Discrete Event Simulation on a Multiprocessor Using Optimistic and Conservative Synchronization
  • The Impact of Lookahead on the Performance of Conservative Distributed Simulation
  • The Role of Knowledge in Distributed Simulation
  • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
  • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
  • Semi-Static Dataflow
  • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor
  • Data Flow on a Queue Machine
  • Short-Packet Transfer Performance in Local Area Rings

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Building Parallel Applications using Design Patterns

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. In Advances in Software Engineering: Topics in Comprehension, Evolution and Evaluation, New York, NY, July 2000. Springer-Verlag. To appear.[7].

Copyright © 2000 by Dhrubajyoti Goswami and Ajit Singh and Bruno Richard Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Research

  • Publications
    • Books
    • Journal Articles
    • Conference Papers
    • Patents
    • Technical Reports
    • Theses
    • Unpublished Manuscripts
  • Presentations

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Using Object-Oriented Techniques for Realizing Parallel Architectural Skeletons

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. In Proc. ISCOPE '99, San Francisco, CA, December 1999[6].

The concept of design patterns has recently emerged as a new paradigm in the context of object-oriented design methodology. Similar ideas are being explored in other areas of computing. In the parallel computing domain, design patterns describe recurring parallel computing problems and their solution strategies. Starting with the late 1980's, several pattern-based systems have been built for facilitating parallel application development. However, most of these systems use patterns in ad hoc manners, thus lacking a generic or standard model for using and intermixing different patterns. This substantially hampers the usability of such systems. Lack of flexibility and extensibility are some of the other major concerns associated with most of these systems. In this paper, we propose a generic (i.e., pattern- and application-independent) model for realizing and using parallel design patterns. The term architectural skeleton is used to represent the application independent, re-usable set of attributes associated with a pattern. The model can provide most of the functionalities of low level message passing libraries, such as PVM or MPI, plus the benefits of the patterns. This results in tremendous flexibility to the user. It turns out that the model is an ideal candidate for an object-oriented style of design and implementation. It is currently implemented as a C++ template-library without requiring any language extension. The generic model, together with the object-oriented and library-based approach, facilitates extensibility. The paper focuses on the design and implementation issues of the model.

Copyright © 1999 by Dhrubajyoti Goswami and Ajit Singh and Bruno Richard Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Issues in Joint Undergraduate Software Engineering Degree Program Design

J. M. Atlee, M. Godfrey, Bruno Richard Preiss, and R. E. Seviora. In Proc. 1999 Canadian Conference on Computer Engineering Education, Vancouver, BC, June 1999. Computer Science Research, Education, and Applications Technology. (Proceedings to appear)[1].

Since early 1998, a joint CS-E&CE undergraduate degree program in software engineering has been under development by the co-authors and their colleagues. This presentation will overview major issues that arise in the design of an undergraduate software engineering degree program. The issues cover the basic delineation of the discipline, curricular content and its distribution, program introduction and maintenance, and impact of internal organizational and external aspects. Selected issues will be discussed in more detail and our perspectives and experience summarized.

Copyright © 1999 by J. M. Atlee, M. Godfrey, B. R. Preiss, and R. E. Seviora.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Architectural Skeletons: The Re-Usable Building-Blocks for Parallel Applications

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. In Proc. 1999 International Conference on Parallel and Distributed Processing Techniques and Applications, volume 3, pages 1250-1256, Las Vegas, NV, June 1999. Computer Science Research, Education, and Applications Technology[5].

Design pattern concepts are being used in the various disciplines of computing. In the parallel computing domain, design patterns specify recurring parallel computing problems and their solution strategies. The paper defines a generic (i.e. application and pattern independent) model for realizing and using parallel design patterns. The term architectural skeleton is coined to represent the application independent, generic set of attributes associated with a pattern. The model is aimed at providing many of the functionalities of MPI, plus the benefits of the patterns. The intent is to provide a considerable amount of flexibility to the user in application development. The generic model also enhances usability. As it turns out, the model is a natural candidate for object-oriented style of design and implementation. It is currently implemented as a C++ template-library without requiring any language extension. The generic model, together with the object-oriented and library-based approach, facilitates extensibility (i.e. new patterns can be added to the system library without requiring any major modifications to the existing system).

Copyright © 1999 by Computer Science Research, Education, and Applications Techology Press.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Design Patterns for the Data Structures and Algorithms Course

Bruno Richard Preiss. In Proc. SIGCSE 1999 Technical Symposium, pages 95-99, New Orleans, LA, March 1999. Association for Computing Machinery, Inc.[41].

Design patterns have recently emerged as a vehicle for describing and documenting recurring object-oriented designs. More significantly, they offer up a long-awaited framework for teaching good software design. This paper espouses the use of object-oriented design patterns in the teaching of the second course in computer science, viz, the data structures and algorithms course.

To use design patterns effectively, it is necessary to present the various data structures and algorithms in a common programming framework. This paper also espouses the use of a single, unified class hierarchy and the committment to a single design throughout the teaching of the second course.

Copyright © 1999 by Association for Computing Machinery, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


The Parsimony Project: A Distributed Simulation Testbed in Java

Bruno Richard Preiss and Carey Ka Wing Wan. In Proc. 1999 International Conference On Web-Based Modelling & Simulation, volume 31 of Simulation Series, pages 89-94, San Francisco, CA, January 1999. Society for Computer Simulation[63].

The Parsimony Project is a vehicle for conducting research in distributed, network-centric computing. The primary objective is the development of a Java-based testbed for distributed discrete-event simulation. In this paper, we present requirements for the implementation of a distributed, discrete-event simulation system based on our earlier research in the area. We show how the Java language and virtual machine support directly these requirements. Finally, we describe briefly a suite comprised of eight different, yet compatible simulators. A user-defined simulation can be run using any of the simulators without modification or even recompilation.

Copyright © 1999 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Memory Management Techniques for Time Warp on a Distributed Memory Machine

Bruno Richard Preiss and Wayne Mervin Loucks. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 30-39, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.[54].

This paper examines memory management issues associated with Time Warp synchronized parallel simulation on distributed memory machines. The paper begins with a summary of the techniques which have been previously proposed for memory management on various parallel processor memory structures. It then concentrates the discussion on parallel simulation executing on a distributed memory computer--a system comprised of separate computers, interconnected by a communications network. An important characteristic of the software developed for such systems is the fact that the dynamic memory is allocated from a pool of memory that is shared by all of the processes at a given processor.

This paper presents a new memory management protocol, pruneback, which recovers space by discarding previous states. This is different from all previous schemes such as artificial rollback and cancelback which recover memory space by causing one or more logical processes to roll back to an earlier simulation time.

The paper includes an empirical study of a parallel simulation of a closed stochastic queueing network showing the relationship between simulation execution time and amount of memory available. The results indicate that using pruneback is significantly more effective than artificial rollback (adapted for a distributed memory computer) for this problem. In the study, varying the memory limits over a 2:1 range resulted in a 1:2 change in artificial rollback execution time and almost no change in pruneback execution time.

Copyright © 1995 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Website of Bruno R. Preiss
next up previous contents external
Next: An Algorithm for Speculative Up: Conference Papers Previous: The Parsimony Project: A

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation

Bruno Richard Preiss. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 11-20, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.[37].

This paper presents an approach for speculative parallel execution of rendezvous-synchronized simulations. Rendezvous-synchronized simulation is based on the notions of processes and gates and on the rendezvous mechanism defined in the the basic process algebra of LOTOS--a standard formal specification language for temporal ordering. Time is introduced via a mechanism similar to the delay behaviour annotation provided by the TOPO toolset.

The algorithm allows speculative gate activations. This increases the available parallelism while ensuring correct execution of the computation.

The model is used to describe closed stochastic queueing network simulations. Analysis of their execution results suggests that the model makes available a promising degree of parallelism.

Copyright © 1995 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


On the Performance of a Multi-Threaded RISC Architecture

Scott K. Lindsay and Bruno Richard Preiss. In Proc. 1993 Canadian Conf. on Elec. and Comp. Eng., volume 1, pages 369-372, Vancouver, B.C., September 1993. Canadian Society for Electrical and Computer Engineering[17].

Multi-threading is a form of parallel processing in which the processor contains several independent contexts which share a single execution pipeline.

We propose a new multi-threaded architecture which differs from previous architectures in that context switches are performed only when the running program cannot execute an instruction in the next cycle. We argue that this strategy can improve pipeline utilization in environments which do not have a large enough number of processes to fully utilize earlier multi-threaded machines.

Copyright © 1993 by Engineering Institute of Canada.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Selecting the Checkpoint Interval in Time Warp Parallel Simulation

Yi-Bing Lin, Bruno Richard Preiss, Wayne Mervin Loucks, and Edward D. Lazowska. In Proc. 1993 Workshop on Parallel and Distributed Simulation, pages 3-10, San Diego, CA, May 1993. Institute of Electrical and Electronics Engineers, Inc.[16].

In Time Warp parallel simulation, a process executes every message as soon as it arrives. If a message with a smaller timestamp subsequently arrives, the process rolls back its state to the time of the earlier message and re-executes from that point.

Clearly, the state of each process must be saved (checkpointed) regularly in case a rollback is necessary. Although most existing Time Warp implementations checkpoint after every state transition, this is not necessary, and the checkpoint interval is in reality a tuning parameter of the simulation.

Lin and Lazowska proposed a model to derive the optimal checkpoint interval by assuming that the rollback behavior of Time Warp is not affected by the frequency of checkpointing. An experimental study conducted by Preiss et al. indicates that the behavior of rollback is affected by the frequency of checkpointing in general, and that the Lin-Lazowska model may not reflect the real situations in general.

This paper extends the Lin-Lazowska model to include the effect of the checkpoint interval on the rollback behavior. The relationship among the checkpoint interval, the rollback behavior, and the overhead associated with state saving and restoration is described. A checkpoint interval selection algorithm which quickly determines the optimal checkpoint interval during the execution of Time Warp simulation is proposed. Empirical results indicate that the algorithm converges quickly and always selects the optimal checkpoint interval.

Copyright © 1993 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Website of Bruno R. Preiss
next up previous contents external
Next: A Unifying Framework for Up: Conference Papers Previous: On the Performance of

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Unifying Framework for Distributed Routing Algorithms

Alan Rooks and Bruno Richard Preiss. In Proc. 1992 Int. Conf. on Parallel Processing, pages I-96-I-100, St. Charles, IL, August 1992. Penn. State University[66].

Distributed routing algorithms presented in the literature have tended to be specific to a particular network topology. In order to investigate the applicability of various high-performance routing strategies to arbitrary topologies, this paper presents a classification framework for distributed routing algorithms. Using this framework, ideas previously presented in a topology-specific context may be applied to new networks. More importantly, however, the classification of distributed routing strategies suggests new areas for investigation: generalizing existing techniques, and examining interactions between techniques. We present some results of such work: significant improvements to routing algorithms for hypercubes, torus networks, star graphs, and odd graphs.

Copyright © 1992 by Pennsylvania State University.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Publications

  • Books
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C#
    • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
    • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java
    • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++
    • Data Structures and Algorithms with Object-Oriented Design Patterns in C++
  • Journal Articles
    • From Design Patterns to Parallel Architecture Skeletons
    • Effects of the Checkpoint Interval on Time and Space in Time Warp
    • Optimal Memory Management for Time Warp Parallel Simulation
    • Simulating Continuous Systems with Piecewise-Linear Signals Using Time Warp
    • Short Packet Transfer Performance in Local Area Ring Networks
    • A Nonuniform Detector Aperture for CT
    • On the Transport of Charged Particles Through Spongy Materials
  • Conference Papers
    • Building Parallel Applications using Design Patterns
    • Using Object-Oriented Techniques for Realizing Parallel Architectural Skeletons
    • Issues in Joint Undergraduate Software Engineering Degree Program Design
    • Architectural Skeletons: The Re-Usable Building-Blocks for Parallel Applications
    • Design Patterns for the Data Structures and Algorithms Course
    • The Parsimony Project: A Distributed Simulation Testbed in Java
    • Memory Management Techniques for Time Warp on a Distributed Memory Machine
    • An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation
    • On the Performance of a Multi-Threaded RISC Architecture
    • Selecting the Checkpoint Interval in Time Warp Parallel Simulation
    • A Unifying Framework for Distributed Routing Algorithms
    • On the Trade-Off between Time and Space in Optimistic Parallel Discrete-Event Simulation
    • Parallel Instance Discrete-Event Simulation Using a Vector Uniprocessor
    • Multi-Threaded Pipelining in a RISC Processor
    • Null Message Cancellation in Conservative Distributed Simulation
    • Performance of Discrete Event Simulation on a Multiprocessor Using Optimistic and Conservative Synchronization
    • The Impact of Lookahead on the Performance of Conservative Distributed Simulation
    • The Role of Knowledge in Distributed Simulation
    • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
    • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
    • Semi-Static Dataflow
    • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor
    • Data Flow on a Queue Machine
    • Short-Packet Transfer Performance in Local Area Rings
  • Patents
    • Method, System and Apparatus For Provisioning Services to Subscribers in a Telecommunications Network
    • Interactive Voice Response System and Method
    • Telecommunication Architecture
    • Strategy for Negotiation of Telecommunication Resources
    • Method and System for Configuring Communication Resources
  • Technical Reports
    • Transputer System User Guide-Second Edition
    • Effects of the Checkpoint Interval on Time and Space in Time Warp
    • Transputer System User Guide
    • YADDES-Yet Another Distributed Discrete Event Simulator: User Manual
    • Prediction and Lookahead in Distributed Simulation
    • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
    • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
    • Semi-static Dataflow
    • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor
  • Theses
    • Data Flow on a Queue Machine
    • Design and Simulation of a Data-Flow Multiprocessor
    • LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks
  • Unpublished Manuscripts
    • A Template-based Model for Developing Parallel Applications on a Network Cluster
    • A Pattern-Based Model for Developing Parallel Applications Using a Network of Processors
    • A Skeleton-Based Model for Developing Parallel Applications Using a Network of Processors
    • An Attributed, Time-Delayed Rendezvous Model for Parallel Discrete Event Simulation
    • Dynamic Rescheduling of Tasks for the Macro-Dataflow Paradigm
    • Ynot Logic Simulator: A Literate C++ Program
    • A Time-Delayed-Rendezvous Model for Parallel Discrete Event Simulation
    • A New Model for Parallel Discrete Event Simulation
    • Parallel Simulation and Lotos: Research In Progress
    • Hobbes: A Multi-Threaded Superscalar Architecture
    • Dynamic Checkpoint Interval Selection in Time Warp Simulation
    • Yaddes-Yet Another Distributed Discrete Event Simulator
    • Queue Machines
    • Effective Memory Bandwidth of Band-Connected Partial Crossbars
    • An Occam Compiler for a Dataflow Multiprocessor (Extended Abstract)

      Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
      Tue Jan 1 13:41:25 EST 2002


On the Trade-Off between Time and Space in Optimistic Parallel Discrete-Event Simulation

Bruno Richard Preiss, Ian Donald MacIntyre, and Wayne Mervin Loucks. In Proc. 1992 Workshop on Parallel and Distributed Simulation, pages 33-42, Newport Beach, CA, January 1992. Society for Computer Simulation[61].

Optimistically synchronized parallel discrete-event simulation is based on the use of communicating sequential processes. Optimistic synchronization means that the processes execute under the assumption that synchronization is fortuitous. Periodic checkpointing of the state of a process allows the process to roll back to an earlier state when synchronization errors occur. This paper examines the effects of varying the frequency of checkpointing on the time and space needed to execute a simulation.

The results presented in this paper were obtained from the simulation of closed stochastic queueing networks with several different topologies. Various process scheduling algorithms and message cancellation strategies are considered. The empirical results are compared with analytical formulae predicting time-optimal checkpoint intervals. It is shown that the time-optimal and space-optimal checkpoint intervals are not the same. Furthermore, a checkpoint interval that is too small adversely affects space more than time; whereas, a checkpoint interval that is too large adversely affects time more than space.

Copyright © 1992 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Parallel Instance Discrete-Event Simulation Using a Vector Uniprocessor

James F. Ohi and Bruno Richard Preiss. In Proc. 1991 Winter Simulation Conf., pages 593-601, Phoenix, AZ, December 1991. Society for Computer Simulation[22].

This paper examines the possibility of running N simulations in parallel on a vector processor. In such a system each instance of execution runs identical code but with a different input data set. The main problem which is addressed is the choice of block selection policy, that is, the choice of which indivisible block of code to execute next. This paper investigates four block selection policies by simulating the execution of such a system. A stochastic flow-graph representation was chosen to model the execution of a simulation. A two-level block selection policy was found to have the best potential speedup of the four block selection policies. The speedup levels achieved were not large, and decreased when there were a large number of unique event types (and therefore handlers) in the simulation system.

Copyright © 1991 by Winter Simulation Conference.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Multi-Threaded Pipelining in a RISC Processor

Ian Donald MacIntyre and Bruno Richard Preiss. In Proc. 1991 Canadian Conf. on Elec. and Comp. Eng., pages 7.3.1-7.3.6, Québec, P.Q., September 1991. Canadian Society for Electrical and Computer Engineering[21].

This paper examines the effects of multithreaded pipelining on the CPI (cycles per instruction) of a RISC processor. The desired CPI in a conventional (single-threaded) RISC processor is one instruction per cycle. However, the CPI is typically more than one because of data hazards, control hazards, and resource hazards in the pipeline.

A multi-threaded processor performs a context switch between every instruction. Multi-threaded pipelining holds out the promise of achieving a lower CPI because it can eliminate data and control hazards, and mask the effects of memory latency. However, multi-threaded pipelining reduces cache hit ratios and requires more chip area to implement.

In this paper, we present a model for predicting the CPI of a multi-threaded pipelined processor. We also present the results of trace-driven simulations of single- and multi-threaded processors. These data show that, for reasonable implementation technologies, and taking into account the chip area penalty, a multi-threaded processor can achieve a lower CPI than a single-threaded processor.

Copyright © 1991 by Engineering Institute of Canada.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Null Message Cancellation in Conservative Distributed Simulation

Bruno Richard Preiss, Wayne Mervin Loucks, Ian Donald MacIntyre, and James A. Field. In Proc. 1991 Workshop on Parallel and Distributed Simulation, pages 33-38, Anaheim, CA, January 1991. Society for Computer Simulation[59].

This paper presents the results of an empirical study of the effects of null message cancellation on the performance of conservatively synchronized distributed simulation. Null message cancellation is an algorithmic modification to the basic conservative synchronization scheme wherein a null message is discarded before receipt when overcome by a message with a larger timestamp.

Empirical results show that the impact of null message cancellation is affected by the lookahead of the logical process (LP). LPs with high inverse lookahead ratios for both customers and null messages benefit the most from null message cancellation.

Two metrics, weighted fractional decrease in overhead and weighted fractional decrease in messages, are used to assess how much of the ``potential'' improvement has been captured by null message cancellation.

Copyright © 1991 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Performance of Discrete Event Simulation on a Multiprocessor Using Optimistic and Conservative Synchronization

Bruno Richard Preiss. In Proc. 1990 Int. Conf. on Parallel Processing, pages 218-222, St.\ Charles, IL, August 1990. Penn. State University[32].

This paper presents an empirical study of the performance of various approaches to distributed discrete event simulation on a message-passing multiprocessor. This study compares the performance of distributed simulation to a sequential simulation of exactly the same system.

Four simulation benchmarks are presented. Each is a closed queueing network with a different topology. Four implementations of each benchmark are described--each with a different lookahead. Each benchmark was run under three customer loads. Each benchmark was run using one, two, four, and eight processors. This paper presents data showing the relationships between speedup, number of processors, lookahead, and computation load.

Copyright © 1990 by Pennsylvania State University.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


The Impact of Lookahead on the Performance of Conservative Distributed Simulation

Bruno Richard Preiss and Wayne Mervin Loucks. In Proc. 1990 European Multiconference--Simulation Methodologies, Languages and Architectures, pages 204-209, Nuremberg, FRG, June 1990. Society for Computer Simulation[53].

It has long been recognized that the ability of a logical process (LP) to ``look ahead'' is crucial to the performance of distributed simulation using conservative synchronization. This paper examines previous definitions of lookahead and lookahead ratio--metrics for characterizing the lookahead ability of LPs.

These metrics do not reflect the use of null messages. In order to assess the impact of null messages on the performance of the simulation, we introduce the concept of null message lookahead and null message inverse lookahead ratio (NILAR). In this paper, we also propose a consistent set of definitions for lookahead and inverse lookahead ratio (ILAR) for customer-neutral systems.

The results discussed in this paper indicate that NILAR acts as a better predictor of simulation speedup than ILAR in the case of closed queueing networks with a large number of null messages. The results presented also indicate that conservatively synchronized distributed simulation implementations must have ILAR values close to one and NILAR values greater than one in order to achieve good speedup. However, ILAR values close to one and NILAR values greater than one do not necessarily result in good speedups. Other requirements are higher loads and limited fanout.

Copyright © 1990 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


The Role of Knowledge in Distributed Simulation

Wayne Mervin Loucks and Bruno Richard Preiss. In Proc. SCS Multiconf. on Distributed Simulation, pages 9-16, San Diego, CA, January 1990. Society for Computer Simulation[20].

This paper examines the suite of closed, stochastic queueing networks proposed by Nicol in order to determine how much impact additional knowledge has on the computation load and the communication load for the multiprocessor running the distributed simulation. Four different networks have been studied at three different levels of traffic intensity. For each network-load pair, four simulation models were constructed, each based on increasing use of user knowledge at a queueing node: no knowledge; service discipline knowledge; queueing discipline knowledge; and routing knowledge. In each case the number of logical process activations and the number of messages used by the distributed simulation was determined and compared to the number of activations needed in a traditional, event-list-driven simulation.

In the area of computation needed, distributed simulation using a conservative synchronization algorithm permits between one half and two thirds of the computation to be eliminated when user knowledge is added to the simulation. The results presented also indicate that the differences caused by the additional knowledge are about twice as dramatic in their impact on communication load as they are on the computation load. The results from using an optimistic synchronization algorithm indicate savings of a similar magnitude, except there is a better reduction in computation than in communication. In all but one case the reduction caused by the additional knowledge is less than an order of magnitude.

Copyright © 1990 by Simulation Councils, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments

Bruno Richard Preiss. In Proc. SCS Multiconf. on Distributed Simulation, pages 139-144, Tampa, FL, March 1989. Society for Computer Simulation[30].

This paper describes a simulation specification language and execution environments that are being used to study the performance of distributed discrete event simulation. First, a simulation programming model based on Chandy-Misra distributed discrete event simulation is presented. Then, the Yaddes simulation specification language is described. Yaddes specifications are translated into C language programs which are then compiled and linked with a run-time support library. Next, the implementation of the run-time support libraries is described. The four libraries currently supported are: (1) traditional, event-list-driven discrete event simulation, (2) distributed discrete event simulation based on multiple, synchronized event lists, (3) Chandy-Misra distributed discrete event simulation, and (4) virtual-time-based distributed discrete event simulation.

Copyright © 1989 by Simulation Councils, Inc.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms

Bruno Richard Preiss, Wayne Mervin Loucks, and V. Carl Hamacher. In Proc. 1988 Winter Simulation Conf., pages 315-324, San Diego, CA, December 1988. Society for Computer Simulation[55].

The main problem associated with comparing distributed discrete event simulation mechanisms is the need to base the comparisons on some common problem specification. This paper presents a specification strategy and language which allows the same simulation problem specification to be used for both distributed discrete event simulation mechanisms as well as the traditional single event list mechanism. This paper includes: a description of the Yaddes specification language; a description of the four simulation mechanisms currently supported; the results for three simulation examples; and an estimate of the performance of a communication structure needed to support the various simulation mechanisms. Currently this work has only been done on a uniprocessor emulating a multiprocessor. This has limited some of our results but lays a significant basis for future simulation mechanism comparison.

Copyright © 1988 by Winter Simulation Conference.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Semi-Static Dataflow

Bruno Richard Preiss and V. Carl Hamacher. In Proc. 1988 Int. Conf. on Parallel Processing, volume 2, pages 127-134, St. Charles, IL, August 1988. Penn. State University[50].

In this paper we present a new dataflow execution model called semi-static dataflow. This model incorporates aspects of conventional static and dynamic dataflow architectures. Programs are partitioned into a collection of dataflow graphs. The execution of each of these graphs is the responsibility of a low-level process called a context. The static dataflow execution model is used to evaluate each of these graphs. Separate instruction and data spaces are used to allow program reentrancy. Function invocation, iteration, and conditional execution are accomplished by dynamically creating new contexts.

The process of creating new contexts and moving data tokens between contexts is called dynamic dataflow graph splicing and is the motivation for calling the whole system semi-static. We present a number of programming paradigms for function invocation, sequential iteration and parallel iteration that are based on dynamic dataflow graph splicing.

We have simulated the execution of a semi-static dataflow multiprocessor. In this paper some of the simulation results obtained for several benchmark programs are presented.

Copyright © 1988 by Pennsylvania State University.

external Full text. external Presentation slides. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Books

  • Data Structures and Algorithms with Object-Oriented Design Patterns in C#
  • Data Structures and Algorithms with Object-Oriented Design Patterns in Java
  • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java
  • Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++
  • Data Structures and Algorithms with Object-Oriented Design Patterns in C++

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor

Bruno Richard Preiss and V. Carl Hamacher. In Proc. 15th Int. Symp. on Computer Architecture, pages 358-364, Honolulu, HA, May 1988. Institute of Electrical and Electronics Engineers, Inc.[48].

This paper describes a scheme for using cache-based hardware to provide simple and efficient message passing support for message-based software systems on a tightly-coupled, shared-bus multiprocessor. This approach is based on the utilization of the existing interprocessor communications medium, the shared bus, to effect the exchange of single-word messages. Communication between processes is accomplished over logical channels using simple, blocking send and receive primitives. The physical processor/channel interface is designed so that the message transfer primitives can be implemented as single machine instructions, namely store and fetch. Special-purpose caches, called message caches, mediate channel operations and effect the exchange of messages over the shared bus.

Copyright © 1988 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Data Flow on a Queue Machine

Bruno Richard Preiss and V. Carl Hamacher. In Proc. 12th Int. Symp. on Computer Architecture, pages 342-351, Boston, MA, August 1985. Institute of Electrical and Electronics Engineers, Inc.[47].

An execution model that supports program reentrancy, recursion, and automatic run-time loop unravelling is described. This execution model is based on queue machines that execute acyclic data-flow graphs. The use of separate instruction and data token spaces allows program reentrancy. Execution environments called contexts execute acyclic data-flow graphs associated with high-level code blocks. Iteration and function activation are implemented by the dynamic creation of contexts and do not require the use of tagged tokens.

A multiprocessor architecture that supports this execution model is proposed. The system architecture is based on a partitioned ring in which each partition of the ring is a conventional processor/memory bus.

The proposed architecture has been simulated in software. A number of test programs have been developed and their execution on the proposed architecture has been evaluated. The performance of the proposed architecture with various numbers of processing elements is described. In addition, a number of task scheduling algorithms are presented and evaluated.

Copyright © 1985 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Short-Packet Transfer Performance in Local Area Rings

Wayne Mervin Loucks, V. Carl Hamacher, Bruno Richard Preiss, and Luke Wong. In Proc. INFOCOM `84, pages 174-184, San Francisco, CA, April 1984. Institute of Electrical and Electronics Engineers, Inc.[18].

The performance of various access control protocols for high-speed, bit-serial, computer communication rings is studied. Local Area Computer Network (LACN) applications in which message packets are of fixed length and shorter than the total inherent propagation delay around the ring are the focus of attention. Token, slotted, and static and dynamic insertion rings are included in the study. In all cases, the transmitting station is responsible for removing its transmitted packet from the ring. Under this type of removal rule, it is possible for the stations to execute their access control algorithms with only a short, fixed, inline delay in each station. The insertion rings dynamically switch longer delays (insertion registers) into the ring when they are transmitting a packet. The transmitter-remove rule operates in such a way that hogging of the ring transmission facility by a subset of stations cannot occur. Expressions that approximate average transfer time as a function of utilization are derived for all ring types and are checked by simulation. For the assumed short-packet environment, it is found that token rings exhibit the slowest transfer times, while dynamic insertion rings are fastest.

Copyright © 1984 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Patents

  • Method, System and Apparatus For Provisioning Services to Subscribers in a Telecommunications Network
  • Interactive Voice Response System and Method
  • Telecommunication Architecture
  • Strategy for Negotiation of Telecommunication Resources
  • Method and System for Configuring Communication Resources

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Method, System and Apparatus For Provisioning Services to Subscribers in a Telecommunications Network

Bruno Richard Preiss and Daniel Wilk, May 2001. Patent Application[65].

The present invention is a telecommunication system, comprising of at least one base station connected to a telecommunication network, a plurality of subscriber stations each cooperating with the base station to establish a connection between a user device operably connected to the subscriber station and the telecommunication network. The base station is also connected with a subscriber database to maintain the subscriber records that contain provisioning information for services, features and account information, and an interactive response system operable to create and modify a subscriber record in the subscriber database in response to input received over the connection from a subscriber using the user device to select services, features and account information for the subscriber station.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Interactive Voice Response System and Method

Bruno Richard Preiss and Daniel Wilk, April 2001. Patent Application[64].

The present invention provides a novel system, method and apparatus of delivering interactive voice response services in a more efficient manner over a network. The system provides for the placement of a subscriber station local to the subscriber and a gateway protocol converter at the base station. The gateway protocol converter converts bandwidth-intensive audio messages into compact data messages, which upon transmission to the subscriber station are converted back into audio messages. By using data messages instead of audio messages over the network, bandwidth is saved on the network for other traffic. Another embodiment of the invention provides a method for transmitting subscriber responses utilizing the system.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Telecommunication Architecture

Michael Stumm, William Martin Snelgrove, Bruno Richard Preiss, Mark Frazer, and Mauricio De Simone, October 2000. Patent Application[68].

The present invention provides a novel telecommunication architecture. The architecture includes a subscriber agent for each subscriber within a telecommunication network. The subscriber agent includes the subscriber's telephone number and calling features. The network also includes terminal agents which are directly associated with one or more terminals such as telephones. The subscriber agent can be associated with one or more terminal agents within the network, so that the subscriber's telephone number and set of calling features can be carried to any terminal within the network. A wide variety of calling features and terminals are supported by the architecture.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Strategy for Negotiation of Telecommunication Resources

Bruno Richard Preiss, March 2000. Patent Application[45].

A telecommunication system and method provides for negotiation between participants in a desired communication to establish the communication. The desired communication is defined by a set of parameters arranged in a hierarchy of stages and values for the parameters of each higher stage are negotiated before negotation of those in the next lower stage. When values for all parameters in all stages have been agreed, the communication is established. If one or more parameter values cannot be agreed at a stage, forensic information is provided to the participants who can retry or abandon the negotations. In one embodiment, a failed negotiation is restarted by a participant modifying a value for a parameter in a previously agreed stage and restarting the negotiation at that stage with the new value. Various negotiating disciplines can be employed to negotiate the stages, including Round Robin negotiating discipline. Further, two or more participants can compete within a stage and the participant with the most favorable terms is selected for inclusion in subsequent stages while the others are removed from the negotiation.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Method and System for Configuring Communication Resources

William Martin Snelgrove, Michael Stumm, Mauricio De Simone, and Bruno Richard Preiss, January 2000. Patent Application[67].

The present invention relates generally to telecommunications, and more specifically to a method and system of providing telecommunication services that is flexible and efficient. Existing Public Switched Telephone Network systems suffer from a number of problems including system complexity, access being proprietary and services being implemented on fixed hardware. As a result, new services are complex and costly to develop, so telcos only provide services that appeal very broadly. The invention provides an open and flexible system by describing communications in terms of graph packets which contain pointers to blocks of software code, rather than containing large blocks of executable code. In the preferred embodiment, these graphs identify software filters which are dedicated to particular tasks, and are based on a standard API. This allows third parties to create new filters and make them available, allowing new services to be provided quickly and easily.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Technical Reports

  • Transputer System User Guide-Second Edition
  • Effects of the Checkpoint Interval on Time and Space in Time Warp
  • Transputer System User Guide
  • YADDES-Yet Another Distributed Discrete Event Simulator: User Manual
  • Prediction and Lookahead in Distributed Simulation
  • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments
  • A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms
  • Semi-static Dataflow
  • A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Data Structures and Algorithms with Object-Oriented Design Patterns in C#

Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in C#. 2001. draft[46].

Preface. 1. Introduction. 2. Algorithm Analysis. 3. Asymptotic Notation. 4. Foundational Data Structures. 5. Data Types and Abstraction. 6. Stacks, Queues and Deques. 7. Ordered Lists and Sorted Lists. 8. Hashing, Hash Tables and Scatter Tables. 9. Trees. 10. Search Trees. 11. Heaps and Priority Queues. 12. Sets, Multisets and Partitions. 13. Garbage Collection. 14. Algorithmic Patterns and Problem Solvers. 15. Sorting Algorithms and Sorters. 16. Graphs and Graph Algorithms. A. C# and Object-Oriented Programming. B. Class Hierarchy Diagrams. C. Character Codes. Index.

Copyright © 2001 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Transputer System User Guide-Second Edition

Bruno Richard Preiss and Neil L. Shipp. CCNG Technical Report E-240, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1995[62].

1. Introduction, 2. Transputer Configurations, 3. Compiling a C Program for a Uniprocessor, 4. Compiling a C++ Program for a Uniprocessor, 5. Compiling a C Program for a Multiprocessor, 6. Compiling a C++ Program for a Multiprocessor, A. batch Manual Page, B. batchcancel Manual Page, C. batchqueue Manual Page, D. icc Manual Page, E. icconf Manual Page, F. iccxx Manual Page, G. icollect Manual Page, H. ilibr Manual Page, I. ilink Manual Page, J. iserver Manual Page, K. mms Manual Page, L. topology Manual Page.

Copyright © 1995 by Computer Communications Networks Group, University of Waterloo.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Effects of the Checkpoint Interval on Time and Space in Time Warp

Bruno Richard Preiss, Wayne Mervin Loucks, and Ian Donald MacIntyre. CCNG Technical Report E-230, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, June 1993[57].

Optimistically synchronized parallel discrete-event simulation is based on the use of communicating sequential processes. Optimistic synchronization means that the processes proceed under the assumption that a synchronized execution schedule is fortuitous. Periodic checkpointing of the state of a process allows the process to roll back to an earlier state when synchronization errors are detected. This paper examines the effects of varying the checkpoint interval on the execution time and memory space needed to perform a parallel simulation.

The empirical results presented in this paper were obtained from the simulation of closed stochastic queueing networks with several different topologies. Various intra-processor process scheduling algorithms and both lazy and aggressive cancellation strategies are considered. The empirical results are compared with analytical formulae predicting time-optimal checkpoint intervals. Two modes of operation, throttling and thrashing have been noted and their effect examined. As the checkpoint interval is increased from one, there is a throttling effect among processes on the same processor which improves performance. When the checkpoint interval is made too large, there is a thrashing effect caused by interaction between processes on different processors. It is shown that the time-optimal and space-optimal checkpoint intervals are not the same. Furthermore, a checkpoint interval that is too small adversely affects space more than time; whereas, a checkpoint interval that is too large adversely affects time more than space.

Copyright © 1993 by Association for Computing Machinery, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Transputer System User Guide

Bruno Richard Preiss. CCNG Technical Report E-212, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1992[33].

1. Introduction, 2. Transputer Configurations, 3. Compiling a C Program for a Uniprocessor, 4. Compiling a C++ Program for a Uniprocessor, 5. Compiling a C Program for a Multiprocessor, 6. Compiling a C++ Program for a Multiprocessor, A. batch Manual Page, B. batchcancel Manual Page, C. batchqueue Manual Page, D. configc Manual Page, E. ilibr Manual Page, F. ilink Manual Page, G. iserver Manual Page, H. mms2 and c004init Manual Page, I. tblock and tclear Manual Page, J. tc, t4c, and t8c Manual Page, K. TCC, t4cc, and t8cc Manual Page.

Copyright © 1992 by Computer Communications Networks Group, University of Waterloo.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


YADDES-Yet Another Distributed Discrete Event Simulator: User Manual

Bruno Richard Preiss and Ian Donald MacIntyre. CCNG Technical Report E-197, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1990[60].

1. Introduction, 2. Computer Simulation and Modelling, 3. Synchronization Methods, 4. Example: Exclusive-OR Circuit, 5. The Yaddes Specification Language, 6. Pseudo-Random Number Generator Package, 7. Statistics Gathering and Reporting Package, 8. Avoiding Deadlock When Using Conservative Synchronization, 9. Example: Queue and Server, A. Distributed Simulation on a Transputer, B. Distributed Simulation on an Apollo Network, C. Yaddes BNF, D. Kernel Statistics, E. Yaddes Compiler Output, F. Pitfalls, G. Yaddes man pages, H. Yaddes Compile-time Error Messages, I. Run-Time Error Messages, Bibliography, Index.

Copyright © 1990 by Computer Communications Networks Group, University of Waterloo.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Prediction and Lookahead in Distributed Simulation

Bruno Richard Preiss and Wayne Mervin Loucks. CCNG Technical Report E-191, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989[52].

This paper presents some of the results of an empirical study of the performance of sequential and distributed discrete event simulations of closed, stochastic queueing networks. We have simulated four different network topologies under various loads. We present results for both conservative and optimistic synchronization methods.

In this paper we focus on the effects of lookahead and prediction on the communication and computation loads in distributed simulation. We have implemented four different versions of each simulation model--each with a different degree of lookahead. We present the lookahead ratio for each of these implementations and show how it is related to the average prediction per message. We argue that average prediction per message is an important parameter for characterizing the performance of distributed simulations. Average prediction per message is affected by network topology, load, lookahead, and the time synchronization method. We present empirical data relating average prediction per message to the communication and computation loads in our simulations.

Copyright © 1989 by Computer Communications Networks Group, University of Waterloo.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments

Bruno Richard Preiss. CCNG Technical Report E-181, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989[31].

This paper describes a simulation specification language and execution environments that are being used to study the performance of distributed discrete event simulation. First, a simulation programming model based on Chandy-Misra distributed discrete event simulation is presented. Then, the Yaddes simulation specification language is described. Yaddes specifications are translated into C language programs which are then compiled and linked with a run-time support library. Next, the implementation of the run-time support libraries is described. The four libraries currently supported are: (1) traditional, event-list-driven discrete event simulation, (2) distributed discrete event simulation based on multiple, synchronized event lists, (3) Chandy-Misra distributed discrete event simulation, and (4) virtual-time-based distributed discrete event simulation.

Copyright © 1989 by Simulation Councils, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Unified Modeling Methodology for Performance Evaluation of Distributed Discrete Event Simulation Mechanisms

Bruno Richard Preiss, Wayne Mervin Loucks, and V. Carl Hamacher. CCNG Technical Report E-180, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989[56].

The main problem associated with comparing distributed discrete event simulation mechanisms is the need to base the comparisons on some common problem specification. This paper presents a specification strategy and language which allows the same simulation problem specification to be used for both distributed discrete event simulation mechanisms as well as the traditional single event list mechanism. This paper includes: a description of the Yaddes specification language; a description of the four simulation mechanisms currently supported; the results for three simulation examples; and an estimate of the performance of a communication structure needed to support the various simulation mechanisms. Currently this work has only been done on a uniprocessor emulating a multiprocessor. This has limited some of our results but lays a significant basis for future simulation mechanism comparison.

Copyright © 1989 by Simulation Councils, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Semi-static Dataflow

Bruno Richard Preiss and V. Carl Hamacher. CCNG Technical Report E-170, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988[51].

In this paper we present a new dataflow execution model called semi-static dataflow. This model incorporates aspects of conventional static and dynamic dataflow architectures. Programs are partitioned into a collection of dataflow graphs. The execution of each of these graphs is the responsibility of a low-level process called a context. The static dataflow execution model is used to evaluate each of these graphs. Separate instruction and data spaces are used to allow program reentrancy. Function invocation, iteration, and conditional execution are accomplished by dynamically creating new contexts.

The process of creating new contexts and moving data tokens between contexts is called dynamic dataflow graph splicing and is the motivation for calling the whole system semi-static. We present a number of programming paradigms for function invocation, sequential iteration and parallel iteration that are based on dynamic dataflow graph splicing.

We have simulated the execution of a semi-static dataflow multiprocessor. In this paper some of the simulation results obtained for several benchmark programs are presented.

Copyright © 1988 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Cache-based Message Passing Scheme for a Shared-bus Multiprocessor

Bruno Richard Preiss and V. Carl Hamacher. CCNG Technical Report E-169, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988[49].

This paper describes a scheme for using cache-based hardware to provide simple and efficient message passing support for message-based software systems on a tightly-coupled, shared-bus multiprocessor. This approach is based on the utilization of the existing interprocessor communications medium, the shared bus, to effect the exchange of single-word messages. Communication between processes is accomplished over logical channels using simple, blocking send and receive primitives. The physical processor/channel interface is designed so that the message transfer primitives can be implemented as single machine instructions, namely store and fetch. Special-purpose caches, called message caches, mediate channel operations and effect the exchange of messages over the shared bus.

Copyright © 1988 by Institute of Electrical and Electronics Engineers, Inc.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Theses

  • Data Flow on a Queue Machine
  • Design and Simulation of a Data-Flow Multiprocessor
  • LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Data Structures and Algorithms with Object-Oriented Design Patterns in Java

Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 2000. 635 pp. ISBN 0-471-34613-6[44].

Preface. 1. Introduction. 2. Algorithm Analysis. 3. Asymptotic Notation. 4. Foundational Data Structures. 5. Data Types and Abstraction. 6. Stacks, Queues and Deques. 7. Ordered Lists and Sorted Lists. 8. Hashing, Hash Tables and Scatter Tables. 9. Trees. 10. Search Trees. 11. Heaps and Priority Queues. 12. Sets, Multisets and Partitions. 13. Garbage Collection. 14. Algorithmic Patterns and Problem Solvers. 15. Sorting Algorithms and Sorters. 16. Graphs and Graph Algorithms. A. Java and Object-Oriented Programming. B. Class Hierarchy Diagrams. C. Character Codes. Index.

Copyright © 2000 by John Wiley & Sons.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Data Flow on a Queue Machine

Bruno Richard Preiss. Data Flow on a Queue Machine. PhD thesis, Department of Electrical Engineering, University of Toronto, 1987. 185 pp.[28].

Data-flow computer architectures have been suggested as a means to achieve increased computational throughput via the automatic detection and exploitation of potential parallelism. In this thesis, a new approach to data-flow, called pseudo-static data flow, is described.

One way of realizing the pseudo-static data-flow approach is to use a queue machine. A queue machine is a processing element that uses a queue as the underlying mechanism for the manipulation of operands. It can be shown that instruction sequences on a queue machine are a natural execution model for computation graphs specified by acyclic data-flow graphs. A compiler for the OCCAM programming language that decomposes source programs into acyclic data-flow graphs for execution on a queue machine is described.

The architecture of a queue-machine processing element intended for use in a multiprocessor system is proposed. A multiprocessor system architecture based on a partitioned bus configuration in a ring topology that uses the queue machine processing element has been simulated in order to characterize its performance potential. The system has been simulated with up to eight processing elements and it exhibits better than linear speed-up as the number of processing elements is increased.

Copyright © 1987 by Bruno R. Preiss.

external Full text. external Presentation slides. external Catalogue Entry. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Design and Simulation of a Data-Flow Multiprocessor

Bruno Richard Preiss. Master's thesis, Department of Electrical Engineering, University of Toronto, 1984. 157 pp.[27].

A data-flow execution model that supports program reentrancy, recursion and automatic run-time loop unraveling without the use of tagged tokens is described. This execution model is based on a novel data-flow program graph representation scheme which is a hybrid of static and dynamic data-flow program representation methods. In particular, the use of separate instruction and data token spaces allows program reentrancy. Execution environments called contexts execute acyclic data-flow graphs associated with high-level code blocks. Iteration and function calling are implemented by the dynamic creation of contexts.

A multiprocessor architecture that supports this execution model is proposed. The system architecture is based on a partitioned ring in which each partition of the ring is a conventional processor/memory bus. Each processor consists of a processing element and a task manager together with a portion of global memory.

The proposed architecture has been simulated in software. A number of test programs have been developed and their execution on the proposed architecture has been evaluated. The performance of the proposed architecture with various numbers of processing elements is described. In addition, a number of task scheduling algorithms are presented and evaluated.

Copyright © 1984 by Bruno R. Preiss.

external Full text. external Catalogue Entry. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks

Bruno Richard Preiss. Bachelor's thesis, Faculty of Applied Science and Engineering--Engineering Science Division (Electrical Engineering Option), University of Toronto, 1982. 122 pp.[26].

LARNS is a powerful computer simulation model for the representation of local area ring networks. A software package that implements the LARNS model has been written. The LARNS model has been designed so that it can represent the various standard ring network protocols, slotted rings, insertion rings and token rings, as well as hybrid schemes. The performance of the TORNET ring, a local area network under developoment at the University of Toronto is analyzed using the LARNS model and software.

In this paper, the LARNS model is described in detail and the software implementation is documented. The TORNET ring is analyzed as a case study, demonstrating the power and flexibility of the LARNS model.

Copyright © 1982 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Unpublished Manuscripts

  • A Template-based Model for Developing Parallel Applications on a Network Cluster
  • A Pattern-Based Model for Developing Parallel Applications Using a Network of Processors
  • A Skeleton-Based Model for Developing Parallel Applications Using a Network of Processors
  • An Attributed, Time-Delayed Rendezvous Model for Parallel Discrete Event Simulation
  • Dynamic Rescheduling of Tasks for the Macro-Dataflow Paradigm
  • Ynot Logic Simulator: A Literate C++ Program
  • A Time-Delayed-Rendezvous Model for Parallel Discrete Event Simulation
  • A New Model for Parallel Discrete Event Simulation
  • Parallel Simulation and Lotos: Research In Progress
  • Hobbes: A Multi-Threaded Superscalar Architecture
  • Dynamic Checkpoint Interval Selection in Time Warp Simulation
  • Yaddes-Yet Another Distributed Discrete Event Simulator
  • Queue Machines
  • Effective Memory Bandwidth of Band-Connected Partial Crossbars
  • An Occam Compiler for a Dataflow Multiprocessor (Extended Abstract)

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


A Template-based Model for Developing Parallel Applications on a Network Cluster

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. manuscript 20 pp., October 1998[4].

Parallel programming is complicated. This complexity arises from the compounding of low-level parallelism related issues with the problems of writing good sequential code. Over the years, various approaches have been proposed to aid parallel program developers. These approaches employ high-level models of parallel computation, thus hiding the low-level parallelism-related details from the user. Different approaches employ different abstraction techniques, such as communication libraries, macros, new parallel languages and abstract data types.

In this paper, we present a template-based approach to parallel application development, which uses frequently occurring patterns for parallelism. A parallel template is a re-usable, application-independent encapsulation of a commonly used parallel computing pattern. It is implemented as a re-usable code-skeleton for quick and reliable development of parallel applications. In the past, parallel programming systems have allowed fast prototyping of parallel applications based on commonly occurring communication and synchronization structures. The uniqueness of this approach is that the templates in this model are generic, with associated structural and behavioral attributes which can be parameterized. Templates have standard interfaces which facilitate their composition.

Unlike the similar approaches in the past, which were mostly suitable for solving a limited subset of parallel applications, this approach provides a systematic development model for the hierarchical development and the subsequent refinements of a vast majority of coarse-grained parallel applications, which can be suitably solved on a network cluster. Two of the main issues addressed are: degree of flexibility in application development and extendibility (hence adaptability) of the development system as per user's need. Both of these issues were some of the major concerns in the past.

Copyright © 1998 by D. Goswami, A. Singh and B. R. Preiss.

external Full text.

Website of Bruno R. Preiss
next up previous contents external
Next: A Pattern-Based Model for Up: Unpublished Manuscripts Previous: Unpublished Manuscripts

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Pattern-Based Model for Developing Parallel Applications Using a Network of Processors

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. Extended Abstract, May 1998[2].

Parallel programming is complicated. This complexity arises from the compounding of low-level parallelism related issues with the problems of writing good sequential code. Over the years, various approaches have been proposed to aid parallel program developers. These approaches employ a high-level model of parallel computation, thus hiding the low-level parallelism-related details from the user. Different approaches employ different abstraction techniques, such as communication libraries, macros, new parallel languages and abstract data types.

In this paper, we present a pattern-based approach to parallel application development, which uses frequently occurring structures for parallelism. A pattern (more precisely, a parallel computing pattern) is a re-usable, application-independent component providing a commonly used parallel structure. It is implemented as a re-usable code-skeleton for quick and reliable development of parallel applications. We believe that many of the parallel applications in use today are composed of at least one of these commonly-used patterns. One or more of these existing patterns can be composed, through a generic hierarchically-defined model and a textual interface, to create the entire parallel application.

The benefit is two-fold: (1) Each pattern is well documented through examples and usages, etc. This provides an user the in-depth knowledge about it. As a result, successive usages of the same pattern in different applications should reduce considerable amount of learning time. (2) The re-usable, application-independent components hide most of the low-level parallelism-related details from the user. These low-level details are often quite tedious and error-prone to implement from scratch. Thus, use of these re-usable components should reduce considerable amount of development time.

Copyright © 1998 by D. Goswami, A. Singh and B. R. Preiss.

external Full text.

Website of Bruno R. Preiss
next up previous contents external
Next: A Skeleton-Based Model for Up: Unpublished Manuscripts Previous: A Template-based Model for

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Skeleton-Based Model for Developing Parallel Applications Using a Network of Processors

Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. manuscript 23 pp., January 1998[3].

One of the greatest obstacles to the mainstream adoption of parallel computing is its complexity. Over the years, various approaches have been proposed to aid parallel program developers. Most of these approaches employ a high-level model of parallel computation, thus hiding the low-level parallelism-related details from the user. Different models employ different abstraction techniques, such as communication libraries, macros, new parallel languages and abstract data types.

In this paper we present a skeleton-based approach which uses frequently occurring structures for parallelism, and is a hybrid of high- and low-level models. Each skeleton is a re-usable, application-independent component providing a commonly used parallel structure. A number of such skeletons can be combined together to create the skeleton of the entire application, which can then be filled in with the application specific components.

Unlike other skeleton-based approaches in the past, this work is unique in the following aspects:

First, it gives a generic definition to a skeleton, with associated structural and behavioral components. The crucial behavioral components were missing in the related works of the past.

Second, it gives a clear-cut and natural model to compose the individual skeletons to develop the entire parallel application. As a result, it is easy for the user to compose skeletons correctly.

Third, unlike the previous approaches, the user can work at various levels of abstraction and also intermix them. For instance, the user can intermix skeletons with the lowest level of communication primitives available to him. This gives him a high degree of flexibility in developing his application.

Fourth, a library-based approach, together with a generic definition of a skeleton, makes it a highly extendible approach, i.e. a new skeleton can be added to the system as per need. Some recent approaches, which intended to be extendible, were in fact hardly extendible due to the absence of a generic viewpoint of a skeleton.

As a direct realization of the model, a library-based development system using object-oriented design methodologies in C++ and the standard Message-Passing Interface (MPI) has been implemented. The latter part of the paper focuses on the implementation and presents experimental results obtained on a cluster of workstations.

Copyright © 1998 by D. Goswami, A. Singh and B. R. Preiss.

external Full text.

Website of Bruno R. Preiss
next up previous contents external
Next: An AttributedTime-Delayed Rendezvous Up: Unpublished Manuscripts Previous: A Pattern-Based Model for

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


An Attributed, Time-Delayed Rendezvous Model for Parallel Discrete Event Simulation

Bruno Richard Preiss. January 1996[39].

This report extends the model for parallel discrete event simulation presented in [37, 36, 34] by introducing the notion attributes (messages). The model is based on the notions of processes and gates and on the rendezvous mechanism defined in the the basic LOTOS process algebra. Time is introduced via a mechanism similar to the delay behaviour annotation provided by the TOPO toolset. Communication between processes is supported by the LOTOS mechanisms of value passing (but not value generation).

An algorithm is presented that allows optimistic gate activations (i.e., speculative computation). This increases the available parallelism while ensuring correct execution of the simulation. This speculative computation can be performed without the expensive overheads associated with checkpointing and rollback recovery.

The model is used to describe two different simulation applications--closed stochastic queue networks simulations and logic simulation. Analysis of preliminary results suggests that the model is successful at making available a degree of speculative parallelism.

Copyright © 1996 by Bruno R. Preiss.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Dynamic Rescheduling of Tasks for the Macro-Dataflow Paradigm

Ka Kay Ho and Bruno R. Preiss. January 1996[11].

A major deficiency of static scheduling techniques is their inability to handle data-dependent looping and conditional branching which cause non-determinismic task execution times, in effect, dynamic alterations of the task graph.

This paper presents a heuristic, hybrid dynamic/static task-scheduling strategy which attempts to handle non-deterministic task execution times. The goal of this strategy is to achieve an improvement in execution time over static scheduling without being overwhelmed by overhead costs. In the proposed approach, the tasks are initially scheduled statically using estimates of task completion times. During run time, tasks are migrated following heuristic rules to take advantage of idle periods on processors. Results from simulations indicate that this technique can produce a performance improvement over the static approach when operating in certain problem domains. The proposed strategy is compared with ``ideal'' static scheduling in order to characterize its efficiency.

Copyright © 1996 by Ka Kay Ho and Bruno R. Preiss.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Ynot Logic Simulator: A Literate C++ Program

Bruno Richard Preiss. 154 pp., July 1995[38].

This book comprises the source code to the [image] (pronounced ``why not?'') logic simlator. It is my first serious foray into the realm of ``literate programming.'' Literate programming was pioneered by Donald Knuth during the development of the TeX typesetting software.

The essential idea is that a literate program is the combination of documentation and source code in a form suitable for reading by human beings. The TeX source to this book and the C++ source code it documents are combined in common files. A literate programming tool which I wrote, called loma, weaves the the files into the TeX source that produced this book and into the C++ source programs which can be compiled to produce the executable program.

The source to this book is not in the public domain. However, the C++ source code for the [image] simulator is available. The C++ source code is cross-referenced with this book, so that you can easily find the page in this which defines and describes the code. The [image] source code can be found on the world-wide web at http://www.pads.uwaterloo.ca/Bruno.Preiss/sources.html.

Copyright © 1995 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java

Bruno Richard Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 1999. 162 pp. ISBN 0-471-36916-0[43].

A complete set of solutions to the exercises in [44].

Copyright © 1999 by John Wiley & Sons.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A Time-Delayed-Rendezvous Model for Parallel Discrete Event Simulation

Bruno Richard Preiss. 1994[36].

This report extends the model for parallel discrete event simulation presented in [34] by introducing the notion of time. The model is based on the notions of processes and gates and on the rendezvous mechanism defined in the the basic LOTOS process algebra. Time is introduced via a mechanism similar to the delay behaviour annotation provided by the TOPO.

An algorithm is presented that allows optimistic gate activations (i.e., speculative computation). This increases the available parallelism while ensuring correct execution of the simulation. In the case where the simulation processes are time-invariant, the optimistic execution admits additional parallelism via the mechanism of time shifting.

The model is used to describe closed stochastic queueing network simulations. Simulations of various queueing network topologies have implemented. Analysis of preliminary results suggests that the model makes available a limited degree of parallelism.

Copyright © 1994 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


A New Model for Parallel Discrete Event Simulation

Bruno Richard Preiss. 1994[34].

This report describes a new model for parallel discrete event simulation based on the notions of processes and gates and on the rendezvous mechanism defined in the the basic LOTOS process algebra. The model presented in this report requires a statically determined fixed number of processes and gates. Whereas it may be possible to simulate many LOTOS specifications using this model, no claim is made regarding the ability of the model to simulate all LOTOS specifications in general.

An execution algorithm is presented that allows optimistic gate activations (i.e., speculative computation). This increases the available parallelism while ensuring correct execution of the simulation. An implementation approach that makes extensive use of bit vectors and bitwise logical operations is described. Finally, a partitioning of the computation for parallel execution is given.

Copyright © 1994 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Parallel Simulation and Lotos: Research In Progress

Bruno Richard Preiss. 1994[35].

No abstract.

Copyright © 1994 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Hobbes: A Multi-Threaded Superscalar Architecture

Bradley John Kish and Bruno Richard Preiss. manuscript 22 pp., July 1994[12].

Several recently proposed microprocessor architectures provide increased performance through out-of-order andor speculative superscalar execution. These architectures are complex and may require changes to the instruction set. Alternatively, multi-threaded architectures attempt to increase pipeline utilization by concurrently executing instructions from different threads. This paper describes a multi-threaded superscalar architecture called Hobbes. In the proposed architecture, up to two instructions can be issued per cycle from any of the four on-board threads. Instructions are issued in-order, without branch prediction. It is argued that using multi-threading increases the available instruction-level parallelism without needing to support complex techniques such as out-of-order issue, register-renaming, branch prediction, and load by-passing of stores.

Results from trace-driven simulations of the architecture are presented. It is shown that, with similar execution, cache and memory resources, Hobbes can provide better performance than a more complex, out-of-order, superscalar processor.

Copyright © 1994 by Bradley J. Kish and Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Dynamic Checkpoint Interval Selection in Time Warp Simulation

Yi-Bing Lin, Bruno Richard Preiss, Wayne Mervin Loucks, and Edward D. Lazowska. manuscript 19 pp.[15].

In Time Warp parallel simulation, a process executes every message as soon as it arrives. If a message with a smaller timestamp subsequently arrives, the process rolls back its state to the time of the earlier message and re-executes from that point.

Clearly, the state of each process must be saved (checkpointed) regularly in case a rollback is necessary. Although most existing Time Warp implementations checkpoint after every state transition, this is not necessary, and the checkpoint interval is in reality a tuning parameter of the simulation.

Lin and Lazowska proposed a model to derive the optimal checkpoint interval by assuming that the rollback behavior of Time Warp is not affected by the frequency of checkpointing. An experimental study conducted by Preiss et al. indicates that the behavior of rollback is affected by the frequency of checkpointing in general, and that the Lin-Lazowska model may not reflect the real situations in general.

This paper extends the Lin-Lazowska model to include the effect of the checkpoint interval on the rollback behavior. The relationship among the checkpoint interval, the rollback behavior, and the overhead associated with state saving and restoration is described. A checkpoint interval selection algorithm which quickly determines the optimal checkpoint interval during the execution of Time Warp simulation is proposed. Empirical results indicate that the algorithm converges quickly and always selects the optimal checkpoint interval.

Copyright © by Yi-Bing Lin and Bruno Richard Preiss and Wayne Mervin Loucks and Edward D. Lazowska.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Yaddes-Yet Another Distributed Discrete Event Simulator

Bruno Richard Preiss. 1988[29].

1. Introduction, 2. Computer Simulation and Modelling, 3. Execution Mechanisms, 4. Example: Exclusive-OR Circuit, 5. The Yaddes Specification Language, 6. Pseudo-Random Number Generator Package, 7. Statistics Gathering and Reporting Package, 8. Avoiding deadlock under the Chandy-Misra execution mechanism, 9. Example: Queue and Server, Appendix 1. Yaddes BNF, Appendix 2. Kernel Statistics from event-list.a, Appendix 3. Kernel Statistics from multi-list[.vax].a, Appendix 4. Kernel Statistics from chandy-misra[.vax].a, Appendix 5. Kernel Statistics from virtual-time[.vax].a, Appendix 6. Yaddes Compiler Output.

Copyright © 1988 by Bruno R. Preiss.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Queue Machines

Bruno R. Preiss. manuscript 10 pp.[24].

Two formal execution models that use a queue as the underlying data structure for the manipulation of operands and results are presented. They are the ``simple queue machine'' and the ``indexed queue machine.'' An algorithm for generating the instruction sequence for evaluating an arbitrary expression on a simple queue machine is presented. It is shown that simple queue machines can efficiently exploit pipelined ALUs and support efficient concurrent expression evaluation.

Acyclic dataflow graphs are shown to naturally lead to the generation of instruction sequences for the indexed queue machine execution model. Since queue machines are statically scheduled, they can exploit the techniques of conventional, Von Neumann architectures such as instruction look-ahead and overlapped execution of sequential instructions, thus overcoming a shortcoming of dataflow execution models. Practical indexed queue machines can be implemented by using registers that act as a cache for the front of the operand queue.

Copyright © by Bruno R. Preiss.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Effective Memory Bandwidth of Band-Connected Partial Crossbars

Bruno R. Preiss. manuscript 17 pp.[23].

This correspondence introduces the class of partially-connected multiple-bus multiprocessor configurations called band-connected partial crossbars. An analytical derivation of the effective memory bandwidth of band-connected partial crossbars, based on a simple model of such systems, is also presented.

Copyright © by Bruno R. Preiss.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


An Occam Compiler for a Dataflow Multiprocessor (Extended Abstract)

Bruno R. Preiss and V. Carl Hamacher. manuscript 10 pp.[25].

In this paper we describe the implementation of an Occam language compiler for a non-traditional architecture. We will describe the design of the internal representation of the Occam source program and outline some of the procedures used in the compilation process. The target processor architecture is a multiprocessor dataflow machine in which the processing elements are optimized for the evaluation of acyclic dataflow graphs. Consequently, the compiler is required to partition the Occam source programs into a collection of acyclic dataflow graphs.

Copyright © by Bruno R. Preiss.

external Full text.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Presentations

  • The Joint CS and E&CE Proposal for a Software Engineering Undergraduate Degree Program. Seminar, PADS Research Group, University of Waterloo. June 1999.
  • Issues in Joint Undergraduate Software Engineering Degree Program Design. Conference Presentation, Canadian Conference on Computer Engineering Education, Vancouver, BC. June 3, 1999.
  • Design Patterns for the Data Structures and Algorithms Course. Conference Presentation, Proc. SIGCSE 1999 Technical Symposium, New Orleans, LA. March 25, 1999.
  • The Parsimony Project: A Distributed Simulation Testbed in Java. Conference Presentation, Proc. 1999 International Conference On Web-Based Modelling & Simulation, San Francisco, CA. January 19, 1999.
  • On the Design and Implementation of the Parsimony Checkpoint Package. Seminar, PADS Research Group, University of Waterloo. January 1999.
  • C++ and Java: Comparing Apples with Oranges. Invited Talk, IEEE Canada, Kitchener-Waterloo Section, University of Waterloo. October 27, 1998.
  • Parsimony: Parallel Simulator Once Named Yaddes. Seminar, PADS Research Group, University of Waterloo. May 1998.
  • Java Inner Classes are Way Cool. Seminar, PADS Research Group, University of Waterloo. March 1998.
  • Tricks of the Masters. Seminar, PADS Research Group, University of Waterloo. 1997.
  • Java from the Perspective of a Long-Time C/C++ Hack. Seminar, PADS Research Group, University of Waterloo. 1997.
  • Preaching/Teaching Objects from Day One. Conference Presentation, Canadian Conference on Computer Engineering Education, Waterloo, ON. May 9, 1997.
  • On the Security of CGI Scripts. Seminar, PADS Research Group, University of Waterloo. 1996.
  • An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation. Conference Presentation, 9th Workshop on Parallel and Distributed Simulation, Lake Placid, NY. June 17, 1995.
  • Experiences in Parallel and Distributed Simulation. Invited Talk, IEEE Student Branch, University of Waterloo. March 14, 1995.
  • Speculative Parallelism in Rendezvous Synchronized Simulation. Research Seminar, Royal Institute of Technology, Stockholm, Sweden. April 22, 1994.
  • The Yaddes Project: Parallel Simulation Research at the University of Waterloo. Research Seminar, Royal Institute of Technology, Stockholm, Sweden. January 31, 1994.
  • Selecting the Checkpoint Interval in Time Warp Simulation. Conference Presentation, 7th Workshop on Parallel and Distributed Simulation, San Diego, CA. May 16, 1993.
  • On the Trade-off betweeen Time and Space in Optimistic Parallel Discrete-Event Simulation. Conference Presentation, 6th Workshop on Parallel and Distributed Simulation, Newport Beach, CA. January 22, 1922.
  • Parallel Simulation of Continuous Systems using Time Warp. Research Seminar, VLSI Research Group Research Review, University of Waterloo. 1991.
  • Null Message Cancellation in Conservative Distributed Simulation. Conference Presentation, 5th Workshop on Parallel and Distributed Simulation, Anaheim, CA. January 23, 1991.
  • Parallel (Distributed) Discrete-Event Simulation. Seminar, Bell Northern Research (BNR), Ottawa, Canada. 1991.
  • Yaddes Project Overview. Seminar, Cadence Computer Corp., Ottawa, Canada. 1990.
  • Simulating Continuous Systems with Piecewise Linear Signals using Time Warp. Research Seminar, VLSI Research Group, University of Waterloo. 1990.
  • Performance of Discrete Event Simulation on a Multiprocessor using Optimistic and Conservative Synchronization. Conference Presentation, 1990 Int. Conf. on Parallel Processing, Pheasant Run, IL. August 17, 1990.
  • The Impact of Lookahead on the Performance of Conservative Distributed Simulation. Conference Presentation, 1990 European Simulation Multiconference, Nuremberg, FRG. June 10, 1990.
  • The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments. Conference Presentation, 3rd Workshop on Parallel and Distributed Simulation, Tampa, FL. March 28, 1989.
  • Potential Harm of Introducing Advanced Computing Concepts in the Lower Years. Seminar, Engineering Education Research Centre (EERC), University of Waterloo. 1989.
  • Progress in Distributed Simulation. Research Seminar, First Annual Information Technology Research Center (ITRC) Research Review, University of Toronto. 1989.
  • A Unified Modeling Methodology for Performance Evaluation Of Distributed Discrete Event Simulation Mechanisms. Conference Presentation, 1988 Winter Simulation Conf., San Diego, CA. December 12, 1988.
  • Semi-Static Dataflow. Conference Presentation, 1988 Int. Conf. on Parallel Processing, Pheasant Run, IL. August 15, 1988.
  • A Distributed Discrete-Event Simulation Testbed. Research Seminar, VLSI Research Group Research Review, VLSI Research Group Research Review. 1988.
  • Dataflow on a Queue Machine. Research Seminar, Department of Electrical Engineering, University of Waterloo. 1987.
  • Dataflow on a Queue Machine. Ph.D. Thesis Oral Defense, Department of Electrical Engineering, University of Toronto. 1987.

Website of Bruno R. Preiss
next up previous contents external
Next: Opuses (Opera?) Up: Research Previous: An Occam Compiler for

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++

Bruno Richard Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 160 pp. ISBN 0-471-32211-3[42].

A complete set of solutions to the exercises in [40].

Copyright © 1999 by John Wiley & Sons.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Personal

  • Home Address
  • CV and Resume

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Tue Jan 1 13:41:25 EST 2002


Home Address

  • external Voice: +1 519 725-2795
  • external Post:

    335 Calvington Place
    Waterloo, ON N2T 1R1
    Canada

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


References

1
J. M. Atlee, M. Godfrey, Bruno Richard Preiss, and R. E. Seviora. Issues in joint undergraduate software engineering degree program design. In Proc. 1999 Canadian Conference on Computer Engineering Education, Vancouver, BC, June 1999. Computer Science Research, Education, and Applications Technology. (Proceedings to appear).

2
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. A pattern-based model for developing parallel applications using a network of processors. Extended Abstract, May 1998.

3
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. A skeleton-based model for developing parallel applications using a network of processors. manuscript 23 pp., January 1998.

4
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. A template-based model for developing parallel applications on a network cluster. manuscript 20 pp., October 1998.

5
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. Architectural skeletons: The re-usable building-blocks for parallel applications. In Proc. 1999 International Conference on Parallel and Distributed Processing Techniques and Applications, volume 3, pages 1250-1256, Las Vegas, NV, June 1999. Computer Science Research, Education, and Applications Technology.

6
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. Using object-oriented techniques for realizing parallel architectural skeletons. In Proc. ISCOPE '99, San Francisco, CA, December 1999.

7
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. Building parallel applications using design patterns. In Advances in Software Engineering: Topics in Comprehension, Evolution and Evaluation, New York, NY, July 2000. Springer-Verlag. To appear.

8
Dhrubajyoti Goswami, Ajit Singh, and Bruno Richard Preiss. From design patterns to parallel architecture skeletons. Journal of Parallel and Distributed Computing, June 2001. (To appear).

9
R. M. Henkelman and Bruno Richard Preiss. A nonuniform detector aperture for CT. Journal of Computer Assisted Tomography, 5(3):401-408, June 1981.

10
R. M. Henkelman and Bruno Richard Preiss. On the transport of charged particles through spongy materials. PhysMed. Biol., 26(6):1151-1156, June 1981.

11
Ka Kay Ho and Bruno R. Preiss. Dynamic rescheduling of tasks for the macro-dataflow paradigm. January 1996.

12
Bradley John Kish and Bruno Richard Preiss. Hobbes: A multi-threaded superscalar architecture. manuscript 22 pp., July 1994.

13
Barry T. W. Kwok and Bruno Richard Preiss. Simulating continuous systems with piecewise-linear signals using Time Warp. International Journal in Computer Simulation, 1(1):59-79, 1991.

14
Yi-Bing Lin and Bruno Richard Preiss. Optimal memory management for Time Warp parallel simulation. ACM Trans. on Modeling and Computer Simulation, 1(4):283-307, October 1991. (Accepted May 1992. Published September 1992.).

15
Yi-Bing Lin, Bruno Richard Preiss, Wayne Mervin Loucks, and Edward D. Lazowska. Dynamic checkpoint interval selection in Time Warp simulation. manuscript 19 pp.

16
Yi-Bing Lin, Bruno Richard Preiss, Wayne Mervin Loucks, and Edward D. Lazowska. Selecting the checkpoint interval in Time Warp parallel simulation. In Proc. 1993 Workshop on Parallel and Distributed Simulation, pages 3-10, San Diego, CA, May 1993. Institute of Electrical and Electronics Engineers, Inc.

17
Scott K. Lindsay and Bruno Richard Preiss. On the performance of a multi-threaded RISC architecture. In Proc. 1993 Canadian Conf. on Elec. and Comp. Eng., volume 1, pages 369-372, Vancouver, B.C., September 1993. Canadian Society for Electrical and Computer Engineering.

18
Wayne Mervin Loucks, V. Carl Hamacher, Bruno Richard Preiss, and Luke Wong. Short-packet transfer performance in local area rings. In Proc. INFOCOM `84, pages 174-184, San Francisco, CA, April 1984. Institute of Electrical and Electronics Engineers, Inc.

19
Wayne Mervin Loucks, V. Carl Hamacher, Bruno Richard Preiss, and Luke Wong. Short packet transfer performance in local area ring networks. IEEE Trans. on Computers, C-34(11):1006-1014, November 1985.

20
Wayne Mervin Loucks and Bruno Richard Preiss. The role of knowledge in distributed simulation. In Proc. SCS Multiconf. on Distributed Simulation, pages 9-16, San Diego, CA, January 1990. Society for Computer Simulation.

21
Ian Donald MacIntyre and Bruno Richard Preiss. Multi-threaded pipelining in a RISC processor. In Proc. 1991 Canadian Conf. on Elec. and Comp. Eng., pages 7.3.1-7.3.6, Québec, P.Q., September 1991. Canadian Society for Electrical and Computer Engineering.

22
James F. Ohi and Bruno Richard Preiss. Parallel instance discrete-event simulation using a vector uniprocessor. In Proc. 1991 Winter Simulation Conf., pages 593-601, Phoenix, AZ, December 1991. Society for Computer Simulation.

23
Bruno R. Preiss. Effective memory bandwidth of band-connected partial crossbars. manuscript 17 pp.

24
Bruno R. Preiss. Queue machines. manuscript 10 pp.

25
Bruno R. Preiss and V. Carl Hamacher. An occam compiler for a dataflow multiprocessor (extended abstract). manuscript 10 pp.

26
Bruno Richard Preiss. LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks. Bachelor's thesis, Faculty of Applied Science and Engineering--Engineering Science Division (Electrical Engineering Option), University of Toronto, 1982. 122 pp.

27
Bruno Richard Preiss. Design and Simulation of a Data-Flow Multiprocessor. Master's thesis, Department of Electrical Engineering, University of Toronto, 1984. 157 pp.

28
Bruno Richard Preiss. Data Flow on a Queue Machine. PhD thesis, Department of Electrical Engineering, University of Toronto, 1987. 185 pp.

29
Bruno Richard Preiss. Yaddes--Yet Another Distributed Discrete Event Simulator. 1988.

30
Bruno Richard Preiss. The Yaddes distributed discrete event simulation specification language and execution environments. In Proc. SCS Multiconf. on Distributed Simulation, pages 139-144, Tampa, FL, March 1989. Society for Computer Simulation.

31
Bruno Richard Preiss. The Yaddes distributed discrete event simulation specification language and execution environments. CCNG Technical Report E-181, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.

32
Bruno Richard Preiss. Performance of discrete event simulation on a multiprocessor using optimistic and conservative synchronization. In Proc. 1990 Int. Conf. on Parallel Processing, pages 218-222, St. Charles, IL, August 1990. Penn. State University.

33
Bruno Richard Preiss. Transputer system user guide. CCNG Technical Report E-212, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1992.

34
Bruno Richard Preiss. A new model for parallel discrete event simulation. 1994.

35
Bruno Richard Preiss. Parallel simulation and lotos: Research in progress. 1994.

36
Bruno Richard Preiss. A time-delayed-rendezvous model for parallel discrete event simulation. 1994.

37
Bruno Richard Preiss. An algorithm for speculative parallel execution of rendezvous-synchronized simulation. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 11-20, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.

38
Bruno Richard Preiss. Ynot Logic Simulator: A Literate C++ Program. 154 pp., July 1995.

39
Bruno Richard Preiss. An attributed, time-delayed rendezvous model for parallel discrete event simulation. January 1996.

40
Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 660 pp. ISBN 0471-24134-2.

41
Bruno Richard Preiss. Design patterns for the data structures and algorithms course. In Proc. SIGCSE 1999 Technical Symposium, pages 95-99, New Orleans, LA, March 1999. Association for Computing Machinery, Inc.

42
Bruno Richard Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 160 pp. ISBN 0-471-32211-3.

43
Bruno Richard Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 1999. 162 pp. ISBN 0-471-36916-0.

44
Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 2000. 635 pp. ISBN 0-471-34613-6.

45
Bruno Richard Preiss. Strategy for negotiation of telecommunication resources, March 2000. Patent Application.

46
Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in C#. 2001. draft.

47
Bruno Richard Preiss and V. Carl Hamacher. Data flow on a queue machine. In Proc. 12th Int. Symp. on Computer Architecture, pages 342-351, Boston, MA, August 1985. Institute of Electrical and Electronics Engineers, Inc.

48
Bruno Richard Preiss and V. Carl Hamacher. A cache-based message passing scheme for a shared-bus multiprocessor. In Proc. 15th Int. Symp. on Computer Architecture, pages 358-364, Honolulu, HA, May 1988. Institute of Electrical and Electronics Engineers, Inc.

49
Bruno Richard Preiss and V. Carl Hamacher. A cache-based message passing scheme for a shared-bus multiprocessor. CCNG Technical Report E-169, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988.

50
Bruno Richard Preiss and V. Carl Hamacher. Semi-static dataflow. In Proc. 1988 Int. Conf. on Parallel Processing, volume 2, pages 127-134, St. Charles, IL, August 1988. Penn. State University.

51
Bruno Richard Preiss and V. Carl Hamacher. Semi-static dataflow. CCNG Technical Report E-170, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988.

52
Bruno Richard Preiss and Wayne Mervin Loucks. Prediction and lookahead in distributed simulation. CCNG Technical Report E-191, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.

53
Bruno Richard Preiss and Wayne Mervin Loucks. The impact of lookahead on the performance of conservative distributed simulation. In Proc. 1990 European Multiconference--Simulation Methodologies, Languages and Architectures, pages 204-209, Nuremberg, FRG, June 1990. Society for Computer Simulation.

54
Bruno Richard Preiss and Wayne Mervin Loucks. Memory management techniques for Time Warp on a distributed memory machine. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 30-39, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.

55
Bruno Richard Preiss, Wayne Mervin Loucks, and V. Carl Hamacher. A unified modeling methodology for performance evaluation of distributed discrete event simulation mechanisms. In Proc. 1988 Winter Simulation Conf., pages 315-324, San Diego, CA, December 1988. Society for Computer Simulation.

56
Bruno Richard Preiss, Wayne Mervin Loucks, and V. Carl Hamacher. A unified modeling methodology for performance evaluation of distributed discrete event simulation mechanisms. CCNG Technical Report E-180, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.

57
Bruno Richard Preiss, Wayne Mervin Loucks, and Ian Donald MacIntyre. Effects of the checkpoint interval on time and space in Time Warp. CCNG Technical Report E-230, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, June 1993.

58
Bruno Richard Preiss, Wayne Mervin Loucks, and Ian Donald MacIntyre. Effects of the checkpoint interval on time and space in Time Warp. ACM Trans. on Modeling and Computer Simulation, 4(3):223-253, July 1994. (Accepted April 1994.).

59
Bruno Richard Preiss, Wayne Mervin Loucks, Ian Donald MacIntyre, and James A. Field. Null message cancellation in conservative distributed simulation. In Proc. 1991 Workshop on Parallel and Distributed Simulation, pages 33-38, Anaheim, CA, January 1991. Society for Computer Simulation.

60
Bruno Richard Preiss and Ian Donald MacIntyre. YADDES--Yet Another Distributed Discrete Event Simulator: User manual. CCNG Technical Report E-197, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1990.

61
Bruno Richard Preiss, Ian Donald MacIntyre, and Wayne Mervin Loucks. On the trade-off between time and space in optimistic parallel discrete-event simulation. In Proc. 1992 Workshop on Parallel and Distributed Simulation, pages 33-42, Newport Beach, CA, January 1992. Society for Computer Simulation.

62
Bruno Richard Preiss and Neil L. Shipp. Transputer system user guide--second edition. CCNG Technical Report E-240, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1995.

63
Bruno Richard Preiss and Carey Ka Wing Wan. The Parsimony project: A distributed simulation testbed in Java. In Proc. 1999 International Conference On Web-Based Modelling & Simulation, volume 31 of Simulation Series, pages 89-94, San Francisco, CA, January 1999. Society for Computer Simulation.

64
Bruno Richard Preiss and Daniel Wilk. Interactive voice response system and method, April 2001. Patent Application.

65
Bruno Richard Preiss and Daniel Wilk. Method, system and apparatus for provisioning services to subscribers in a telecommunications network, May 2001. Patent Application.

66
Alan Rooks and Bruno Richard Preiss. A unifying framework for distributed routing algorithms. In Proc. 1992 Int. Conf. on Parallel Processing, pages I-96-I-100, St. Charles, IL, August 1992. Penn. State University.

67
William Martin Snelgrove, Michael Stumm, Mauricio De Simone, and Bruno Richard Preiss. Method and system for configuring communication resources, January 2000. Patent Application.

68
Michael Stumm, William Martin Snelgrove, Bruno Richard Preiss, Mark Frazer, and Mauricio De Simone. Telecommunication architecture, October 2000. Patent Application.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Data Structures and Algorithms with Object-Oriented Design Patterns in C++

Bruno Richard Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 660 pp. ISBN 0471-24134-2.[40].

Preface. 1. Introduction. 2. Algorithm Analysis. 3. Asymptotic Notation. 4. Foundational Data Structures. 5. Data Types and Abstraction. 6. Stacks, Queues and Deques. 7. Ordered Lists and Sorted Lists. 8. Hashing, Hash Tables and Scatter Tables. 9. Trees. 10. Search Trees. 11. Heaps and Priority Queues. 12. Sets, Multisets and Partitions. 13. Dynamic Storage Allocation. 14. Algorithmic Patterns and Problem Solvers. 15. Sorting Algorithms and Sorters. 16. Graphs and Graph Algorithms. A. C++ and Object-Oriented Programming. B. Class Hierarchy Diagrams. C. Character Codes. List of Programs. List of Figures. List of Tables. Index.

Copyright © 1999 by John Wiley & Sons.

external Full text. external BibTeX entry.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Tue Jan 1 13:41:25 EST 2002


Bruno R. Preiss-Signature Page

Bruno R. Preiss, B.A.Sc., M.A.Sc., Ph.D., P.Eng.
mail 335 Calvington Place
Waterloo, ON N2T 1R1
Canada

network Email: brpreiss@brpreiss.com
URL: http://www.brpreiss.com

Copyright © 1994-2002 by Bruno R. Preiss, P.Eng. All rights reserved.


404 Not Found

Not Found

The requested URL /books/body.html was not found on this server.

Apache/1.3.14 Server at www.brpreiss.com Port 80


404 Not Found

Not Found

The requested URL /books/sidebar.html was not found on this server.

Apache/1.3.14 Server at www.brpreiss.com Port 80


Copyright Notice

Copyright © 1999 by Bruno R. Preiss.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the author.

Bruno
Copyright © 1999 by Bruno R. Preiss, P.Eng. All rights reserved.


Bruno R. Preiss-Signature Page

Bruno R. Preiss, B.A.Sc., M.A.Sc., Ph.D., P.Eng.
Associate Professor
Department of Electrical and Computer Engineering
University of Waterloo
Waterloo, ON N2L 3G1
Canada

Email: brpreiss@brpreiss.com
URL: http://www.brpreiss.com

Copyright © 1999 by Bruno R. Preiss, P.Eng. All rights reserved.


 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Data Structures and Algorithms
with Object-Oriented Design Patterns in C++

Bruno R. Preiss
B.A.Sc., M.A.Sc., Ph.D., P.Eng.
Associate Professor
Department of Electrical and Computer Engineering
University of Waterloo, Waterloo, Canada

  • Colophon
  • Dedication
  • Preface
  • Contents
  • Introduction
  • Algorithm Analysis
  • Asymptotic Notation
  • Foundational Data Structures
  • Data Types and Abstraction
  • Stacks, Queues and Deques
  • Ordered Lists and Sorted Lists
  • Hashing, Hash Tables and Scatter Tables
  • Trees
  • Search Trees
  • Heaps and Priority Queues
  • Sets, Multisets and Partitions
  • Dynamic Storage Allocation: The Other Kind of Heap
  • Algorithmic Patterns and Problem Solvers
  • Sorting Algorithms and Sorters
  • Graphs and Graph Algorithms
  • C++ and Object-Oriented Programming
  • Class Hierarchy Diagrams
  • Character Codes
  • References
  • Index
next up previous contents index

Bruno Copyright © 1997 by Bruno R. Preiss, P.Eng. All rights reserved.


Footnotes

...102#102.
The notation 103#103 denotes the floor function , which is defined as follows: For any real number x, 104#104 is the greatest integer less than or equal to x. While we are on the subject, there is a related function, the ceiling function , written 105#105. For any real number x, 106#106 is the smallest integer greater than or equal to x.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...121#121.
In fact, we would normally write 122#122, but we have not yet seen the 1#1 notation which is introduced in Chapter gif .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...rule
Guillaume François Antoine de L'Hôpital, marquis de Sainte-Mesme, is known for his rule for computing limits which states that if 360#360 and 361#361, then

362#362

where f'(n) and g'(n) are the first derivatives with respect to n of f(n) and g(n), respectively. The rule is also effective if 363#363 and 364#364.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...commensurate.
Functions which are commensurate are functions which can be compared one with the other.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...439#439.
This notion of the looseness (tightness ) of an asymptotic bound is related to but not exactly the same as that given in Definition gif .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...numbers.
Fibonacci numbers are named in honor of Leonardo Pisano (Leonardo of Pisa), the son of Bonaccio (in Latin, Filius Bonaccii), who discovered the series in 1202.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...numbers.
These running times were measured on a Sun SPARCstation 5, Model 85, which has an 85 MHz clock, and 32MB RAM. The programs were compiled using the SPARCompiler C++ 4.1 compiler, and run under the Solaris 2.3 operating system. The times shown are user CPU times, measured in seconds.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...T.
This is not an unrealistic assumption. In the absence of a user-defined overloading of the assignment operator, the default behavior for assignment is to copy one-by-one the data members of the object of type T.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=2789>
The outofrange exception is defined in the C++ standard library.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...616#616.
Actually, in C++ it is not possible to have a data type T for which 617#617. While C expressly forbids an empty struct declaration, i.e., one which contains no data members (fields), such declarations are permissible and quite common in C++ programs. However, the draft standard specifically says that even though an empty struct or class declaration is legal, it shall always be the case that 618#618!
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=3616>
The domainerror exception is defined in the C++ standard library.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...represents.
The address attribute is sometimes called its l-value and the value attribute is sometimes called is r-value . This terminology arises from considering the semantics of an assignment statement such as y = x. The meaning of such as statement is ``take the value of variable x and store it in memory at the address of variable y.'' So, when a variable appears on the right-hand-side of an assignment, we use its r-value; and when it appears on the left-hand-size, we use its l-value.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...address,
Actually, it is possible for two variables to occupy the same memory location if a union is used. While there are legitimate uses for unions (see Chapter gif ), it is reasonable to assume here that all objects have unique addresses.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=8022> .
The word deque is usually pronounced like ``deck'' and sometimes like ``deek.''
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...class.
I will admit that the name ListAsLinkedList is somewhat confusing. However, it is a whole word shorter than OrderedListAsLinkedList and that much easier to type!
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...order
A total order is a relation, say <, defined on a set of elements, say 842#842, with the following properties:
  1. For all pairs of elements 843#843, such that 844#844, exactly one of either i<j or j<i holds. (All elements are commensurate ).
  2. For all triples 845#845, 846#846. (The relation 397#397 is transitive ).

(See also Definition gif ).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...

867#867

This is the Swedish word for the number two. Since there is no å in the ASCII character set, for the purposes of the discussion in this chapter we will use the ASCII code for a in its place. However, the Swedish national variant of the ISO 646 character set uses the code corresponding to the ASCII character ``}''.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...

867#867

I have been advised that a book with out sex will never be a best seller. ``Sex'' is the Swedish word for the number six.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...integer.
The function bitsizeof can be implemented as the C++ preprocessor macro #define bitsizeof(T) (8*sizeof(T)) which determines the number of bits required to represent the type T assuming eight-bit bytes.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...prime
Two numbers x and y are relatively prime if there is no number other than one that divides both x and y evenly.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...type.
Typically, on 32-bit computer 925#925.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...i.
What else would it be?
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=15292> .
Isomorphic is a fancy word that means being of identical or similar form or shape or structure.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...div.
Of course, in C++ we can overload the built-in operators. However, having done so we would write an infix expression rather than the prefix one.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...Landis
Russian mathematicians G. M. Adel'son-Vel'skiı and E. M. Landis published this result in 1962.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...trees.
Obviously since B-Trees are M-way trees, the ``B'' in B-Tree does not stand for binary. B-Trees were invented by R. Bayer and E. McCright in 1972, so the ``B'' either stands for balanced or Bayer-take your pick.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...HREF="page385.html#exercisepqueuesbinom"> gif ).
Isaac Newton discovered the binomial theorem in 1676 but did not publish a proof. Leonhard Euler attempted a proof in 1774. Karl Friedrich Gauss produced the first correct proof in 1812.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...subsets.
Stirling numbers of the second kind are given by the formula

1655#1655

where n>0 and 1656#1656.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...p.
Actually, the programmer must be careful when doing this kind of thing. Because C++ cannot guarantee the order in which globals declared in separate files are constructed, in some circumstances it may be possible for operator new and operator delete to be called before the storage pool p has been initialized by its constructor. The solution to this is to use a technique such as double construction [33]. Such trickery is beyond the scope of this text. Caveat emptor.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...Log2Ceil.
We assume that there exists the function unsigned int Log2Ceil (unsigned int n); which computes 1782#1782. It is possible to compute this function in constant time using simple bit manipulations. Most computers have instructions that make the implementation trivial.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...space.
The reader may find it instructive to compare Program gif with Program gif and Program gif .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...space.
The reader may find it instructive to compare Program gif with Program gif and Program gif .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...class.
The declaration of the class itself has been omitted since it follows directly from Programs gif and gif .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=33394> .
The table is named in honor of Blaise Pascal who published a treatise on the subject in 1653.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...number!
Prime numbers of the form 2015#2015 are known as Mersenne primes .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...2018#2018.
For convenience, we use the notation 2019#2019 to denote 2020#2020.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...time.
In this chapter we assume that 601#601 and that 1296#1296.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...NAME=35375> .
Unfortunately, the fame of bubble sort exceeds by far its practical value.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...zero.
There is also the symmetrical case in which i is always n-1.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.


Colophon

Copyright © 1997 by Bruno R. Preiss.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the author.

This book was prepared with LaTeX and reproduced from camera-ready copy supplied by the author. The book is typeset using the Computer Modern fonts designed by Donald E. Knuth with various additional glyphs designed by the author and implemented using METAFONT.

METAFONT is a trademark of Addison Wesley Publishing Company.

SPARCstation, Solaris, and Java are registered trademarks of Sun Microsystems.

TeX is a trademark of the American Mathematical Society.

UNIX is a registered trademark of AT&T Bell Laboratories.


Introduction

  • What This Book Is About
  • Object-Oriented Design
  • Object Hierarchies and Design Patterns
  • The Features of C++ You Need to Know
  • How This Book Is Organized


Multi-Dimensional Arrays

A multi-dimensional array of dimension n (i.e., an n-dimensional array or simply n-D array) is a collection of items which is accessed via n subscript expressions. E.g., the [image] element of the two-dimensional array x is accessed by writing x[i][j].

The C++ programming language provides built-in support for multi-dimensional arrays. However, the built-in multi-dimensional arrays suffer the same indignities that simple one-dimensional arrays do: Arrays in C++ are not first-class data types. There is no such thing as an array-valued expression. Consequently, you cannot use an array as an actual value parameter of a function; you cannot return an array value from a function; and, you cannot assign one array to another. The subscript ranges all start at zero and there is no bounds checking of array subscript expressions. Finally, the size of an array is static and fixed at compile time, unless dynamic memory allocation is explicitly used by the programmer.

In order to illustrate how these deficiencies of the C++ built-in multi-dimensional arrays can be overcome, we will examine the implementation of a two-dimensional array class, Array2D, that is based on the one-dimensional array class discussed in Section gif .

  • Array Subscript Calculations
  • Two-Dimensional Array Implementation
  • Multi-Dimensional Subscripting in C++
  • Canonical Matrix Multiplication


Array Subscript Calculations

The memory of a computer is essentially a one-dimensional array--the memory address is the array subscript. Therefore, the most natural way to implement a multi-dimensional array is to store its elements in a one-dimensional array. In order to do this, we need a mapping from the n subscript expressions used to access an element of the multi-dimensional array to the one subscript expression used to access the one-dimensional array. E.g., suppose we have a two-dimensional array of elements of type T, T a[2][3], the elements of which are to be stored in a one-dimensional array, T b[20]. Then we need to determine which element of b, say b[k], will be accessed given a reference of the form a[i][j]. I.e., we need the mapping f such that [image].

The mapping function determines the way in which the elements of the array are stored in memory. The most common way to represent an array is in row-major order , also known as lexicographic order . E.g., consider the 2D array T a[2][3]. The row-major layout of this array is shown in Figure gif .

figure3774
Figure: Row-Major Order Layout of a 2D Array

In row-major layout, it is the right-most subscript expression (the column index) that increases the fastest. As a result, the elements of the rows of the matrix end up stored in contiguous memory locations. In Figure gif , the array is stored starting from address a. The first element of the first row is a address [image]. The first element of the second row is at address [image], since there are 3 elements in each row.

We can now generalize this to an arbitrary n-dimensional array. Suppose we have an n-D array declared as T a[ ][ ][ ][ ]. Furthermore, let a be the starting address of the array. Then, the address of the element a[ ][ ][ ][ ] is given by

where

The running time required to calculate the address appears to be [image] since the address is the sum of n terms and for each term we need to compute [image], which requires O(n) multiplications in the worst case. However, the address calculation can in fact be done in O(n) time using the following algorithm:

unsigned int product = 1;
T* address =    ;
for (int j = n; j >= 1; -j)

    address += product *    ;
    product *=    ;

This algorithm makes subtle use of the way that address arithmetic is done in C++. Since the variable address is of type T*, it is not necessary to scale the computation by sizeof(T). In C++ whenever an integer value is added to a pointer variable, it is automatically scaled by the compiler before the addition.


Two-Dimensional Array Implementation

In this section we illustrate the implementation of a multi-dimensional array by giving the code for a generic two-dimensional array class, Array2D<T>, which is derived from the Array<T> class discussed in Section gif . The declaration of the Array2D<T> class template is shown in Program gif .

program4055
Program: Array2D<T> and Array2D<T>::Row Class Definitions

Objects of the Array2D<T> class contain three member variables--numberOfRows, numberOfColumns, and array. The first two record the dimensions of the array. The last is an instance of the one-dimension array object discussed in Section gif .

The definitions of the Array2D<T> class member functions are given in Program gif . The constructor takes two arguments, m and n, which are the desired dimensions of the array. It calls the Array<T> class constructor to build a one-dimensional array of size mn. Using the result from Section gif , it can be shown that the running time for the Array2D<T> constructor is [image].

program4087
Program: Array2D<T> Class Member Functions

The Select function takes two arguments, i and j, and returns a reference to the [image] element of the array. In the previous section we saw that the running time for the array subscripting calculation in an k-dimensional array is O(k). For a two-dimensional array, k=2. Therefore, the running time for the subscript calculation is O(2)=O(1).


Multi-Dimensional Subscripting in C++

When accessing the elements of a multi-dimensional array, programmers usually prefer to use the C++ array subscripting operator rather than call a member function explicitly. This is because it is much more convenient to write a[i][j] than to write a.Select(i,j). However, C++ does not directly support the overloading of multi-dimensional array subscripting. E.g., an experienced Fortran programmer would expect to be able to write a[i,j] and to overload operator[](int,int). Alas, neither of these things is valid in C++.

The solution to this problem is to do the subscripting in two steps. Consider the reference a[i][j]. In C++ this is equivalent to

a.operator[] (i).operator[] (j)
In effect, the first subscripting operator selects the [image] row, and then the second subscripting operator picks the [image] element of that row.

Program gif gives a framework for implementing the two-step process in C++. Two object classes are declared--Array2D<T> and the nested class Row. The latter class is used to represent a reference to a particular row of a given two-dimensional array.

In Program gif we see that operator[] for Array2D<T> objects returns an object of type Array2D<T>::Row. The purpose of a Row object is to ``remember'' both the array and the row of that array that is being accessed--its data members are a reference to the accessed array, and the row number.

There is also a subscripting operator defined for Row class objects. This one uses the remembered row number together with the given column number to call the Select function on the appropriate array.

The definition of the Array2D<T> subscripting member function, operator[], is given in Program gif . This function simply constructs an instance of the Array2D<T>::Row class. Clearly, the running time of this function is O(1).

Given the Array2D<T> class we are able to write code such as the following:

Array2D<int> a (4, 5);
for (int i = 0; i < 4; ++i)
    for (int j = 0; j < 5; ++j)
        a[i][j] = 0;
which declares a [image] array of integers, and initializes all of the elements with the value zero.


Canonical Matrix Multiplication

Given an [image] matrix A and an [image] matrix B, the product C=AB is an [image] matrix. The elements of the result matrix are given by

Accordingly, in order to compute the produce matrix, C, we need to compute mp summations each of which is the sum of n product terms.

To represent matrices we use Matrix<T> class shown in Program gif . This class simply extends the Array2D<T> class by adding declarations for the various operations on matrices.

program4136
Program: Matrix<T> Class Definition

An algorithm to compute the matrix product is given in Program gif . The algorithm given is a direct implementation of Equation gif .

program4153
Program: Matrix<T> Class Multiplication Operator Definition

The matrix multiplication routine in Program gif overloads the multiplication operator, operator*, as a member function of the Matrix<T> class. As a result it takes a single argument, arg, which is a const reference to a matrix. The routine computes a result matrix which is the produce of *this and arg.

In C++ there are two ways to overload operator*. The first is to define a non-member function

Matrix<T> operator * (Matrix<T> const&, Matrix<T> const&);
which takes two arguments, both of them const references to Matrix<T> objects, and produces a result of type Matrix<T>. The second way is to declare a const member function of class Matrix<T> called
Matrix<T> Matrix<T>::operator * (Matrix<T> const&) const;
which takes as its single argument a const reference to a Matrix<T> class object.

There are pros and cons associated with each approach. The former approach requires the function to be declared a friend of the Matrix<T> class and any of the classes from which it may be derived if access to the protected members of those classes is needed in the implementation. On the other hand, in this approach, the operands are treated symmetrically with respect to implicit type coercion .

The latter approach has been chosen for the implementation of the matrix multiplication operator since Matrix<T> is a derived class and the implementation of the operator requires access to protected members of the base class.

To determine the running time of Program gif , we need to make some assumptions about the running times for assignment, addition, and multiplication of objects of type T. For simplicity, we shall assume that these are all constant. We will also assume that the dimensions of the matrices to be multiplied are [image] and [image].

The algorithm begins by checking to see that the matrices to be multiplied have compatible dimensions. I.e., the number of columns of the first matrix must be equal to the number of rows of the second one. This check takes O(1) time in the worst case.

Next a matrix in which the result will be formed is constructed (line 6). The running time for this is [image]. If T is one of the built-in types whose constructor does nothing, this running time reduces to O(1).

For each value of i and j, the innermost loop (lines 12-13) does n iterations. Each iteration takes a constant amount of time, assuming assignment, addition, and multiplication of type T objects each take a constant amount of time.

The body of the middle loop (lines 9-15) takes time O(n) for each value of i and j. The middle loop is done for p iterations, giving the running time of O(np) for each value of i. Since, the outer loop does m iterations, its overall running time is O(mnp).

Finally, the result matrix is returned on line 17. The return involves a call to the copy constructor for Matrix<T> class objects. In fact, no such constructor has been declared for Matrix<T> class objects. The default behavior of C++ in this case is to create a default copy constructor for Matrix<T> class objects. In C++ the behavior of the default copy constructor for objects of class X is to call the copy constructor for the each of the base classes from which the class X may be derived and then to copy the data members of X one-by-one using their respective copy constructors. Since the Matrix<T> class is derived from the Array2D<T> class and since the Matrix<T> class has no data members, the running time of this default copy constructor is equal to the running time of the copy constructor for Array2D<T> class objects (see Section gif ). Hence the running time of line 18 is [image]. Assuming that [image], we get O(mp).

In summary, we have shown that lines 4-5 are O(1); line 6 is O(mp); lines 7-16 are O(mnp); and line 17 is O(mp). Therefore, the running time of the canonical matrix multiplication algorithm is O(mnp).


Exercises

    1. How much space does the Array<T> class declared in Program gif use for an array of integers of length N?
    2. How much space does the LinkedList<T> class declared in Program gif use to store a list of n integers?
    3. For what value of N/n do the two classes use the same amount of space?
  2. The array subscripting operators defined in Program gif only test whether [image], the do not test whether [image]. Explain why the second test is not required in this implementation.
  3. The SetBase member function of the Array<T> class defined in Program gif simply changes the value of the base member variable. As a result, after the base is changed, all the array elements appear to have moved. How might the routine be modified so that the elements of the array don't change their apparent locations when the base is changed?
  4. Write the C++ code for the assignment operator of the Array<T> class declared in Program gif .
  5. Which routines are affected if we drop the tail member variable from the LinkedList<T> class declared in Program gif ? Determine new running times for the affected routines.
  6. How does the implementation of the Prepend function of the LinkedList<T> class defined in Program gif change when a circular list with a sentinel is used as shown in Figure gif (c).
  7. How does the implementation of the Append function of the LinkedList<T> class defined in Program gif change when a circular list with a sentinel is used as shown in Figure gif (c).
  8. Consider the assignment operator for the LinkedList<T> class given in Program gif . What is the purpose of the test &linkedlist != this on line 15?
  9. Equation gif is only correct if the subscript ranges in each dimension start at zero. How does the formula change when each dimension is allowed to have an arbitrary subscript range?
  10. The alternative to row-major layout of of multi-dimensional arrays is called column-major order . In column-major layout the leftmost subscript expression increases fastest. For example, the elements of the columns of a two-dimensional matrix end up stored in contiguous memory locations. Modify Equation gif to compute the correct address for column-major layout.
  11. We wish to add an operator+ member function to the Matrix<T> class declared in Program gif that does the usual matrix addition. Write the C++ code for this member function.


Projects

  1. Complete the implementation of the Array<T> class declared in Program gif . Write a test suite to verify all of the functionality. Try to exercise every line of code in the implementation.
  2. Complete the implementation of the LinkedList<T> class declared in Program gif . Write a test suite to verify all of the functionality. Try to exercise every line of code in the implementation.
  3. Change the implementation of the LinkedList<T> class declared in Program gif by removing the tail member variable. I.e., implement the singly-linked list variant shown in Figure gif (a). Write a test suite to verify all of the functionality. Try to exercise every line of code in the implementation.
  4. Change the implementation of the LinkedList<T> class declared in Program gif so that it uses a circular, singly-linked list with a sentinel as shown in Figure gif (c). Write a test suite to verify all of the functionality. Try to exercise every line of code in the implementation.
  5. The Array2D<T> class declared in Program gif only supports subscript ranges starting at zero. Modify the implementation to allow an arbitrary subscript base in each dimension.
  6. Design an implement a three-dimensional array class Array3D<T> based on the two-dimensional class Array2D<T> declared in Program gif .
  7. A row vector is a [image] matrix and a column vector is an [image] matrix. Define and implement classes RowVector<T> and ColumnVector<T> as classes derived from the base class Array2D<T>.


Data Types and Abstraction

It is said that ``computer science is [the] science of abstraction[2].'' But what exactly is abstraction? Abstraction is ``the idea of a quality thought of apart from any particular object or real thing having that quality''[9]. For example, we can think about the size of an object without knowing what that object is. Similarly, we can think about the way a car is driven without knowing its model or make.

Abstraction is used to suppress irrelevant details while at the same time emphasizing relevant ones. The benefit of abstraction is that it makes it easier for the programmer to think about the problem to be solved.

  • Abstract Data Types
  • Design Patterns
  • Exercises
  • Projects


Abstract Data Types

A variable in a procedural programming language such as Fortran , Pascal , C , and C++ , is an abstraction. The abstraction comprises a number of attributes --name , address , value , lifetime , scope , type , and size . Each attribute has an associated value. E.g., if we declare an integer variable in C++, int x, we say that the name attribute has value ``x'' and that the type attribute has value ``int''.

Unfortunately, the terminology can be somewhat confusing: The word ``value'' has two different meanings--in one instance it denotes one of the attributes and in the other it denotes the quantity assigned to an attribute. E.g., after the assignment statement x = 5, the value attribute has the value five.

The name of a variable is the textual label used to refer to that variable in the text of the source program. The address of a variable denotes is location in memory. The value attribute is the quantity which that variable represents. gif The lifetime of a variable is the interval of time during the execution of the program in which the variable is said to exist. The scope of a variable is the set of statements in the text of the source program in which the variable is said to be visible . The type of a variable denotes the set of values which can be assigned to the value attribute and the set of operations which can be performed on the variable. Finally, the size attribute denotes the amount of storage required to represent the variable.

The process of assigning a value to an attribute is called binding . When a value is assigned to an attribute, that attribute is said to be bound to the value. Depending on the semantics of the programming language, and on the attribute in question, the binding may be done statically by the compiler or dynamically at run-time. E.g., in C++ the type of a variable is determined at compile time--static binding . On the other hand, the value of a variable is usually not determined until run-time--dynamic binding .

In this chapter we are concerned primarily with the type attribute of a variable. The type of a variable specifies two sets:

  • a set of values; and,
  • a set of operations.
For example, when we declare a variable, say x, of type int, we know that x can represent an integer in the range [image] (assuming 32-bit integers) and that we can perform operations on x such as addition, subtraction, multiplication, and division.

The type int is an abstract data type in the sense that we can think about the qualities of an int apart from any real thing having that quality. In other words, we don't need to know how ints are represented nor how the operations are implemented to be able to be able to use them or reason about them.

In designing object-oriented programs, one of the primary concerns of the programmer is to develop an appropriate collection of abstractions for the application at hand, and then to define suitable abstract data types to represent those abstractions. In so doing, the programmer must be conscious of the fact that defining an abstract data type requires the specification of both a set of values and a set of operations on those values.

Indeed, it has been only since the advent of the so-called object-oriented programming languages that the we see programming languages which provide the necessary constructs to properly declare abstract data types. E.g., in C++, the class construct is the means by which both a set of values and an associated set of operations is declared. Compare this with the struct construct of C or Pascal's record , which only allow the specification of a set of values!


Design Patterns

An experienced programmer is in a sense like concert musician--she has mastered a certain repertoire of pieces which she is prepared to play at any time. For the programmer, the repertoire comprises a set of abstract data types with which she is familiar and which she is able to use in her programs as the need arises.

The chapters following this present a basic repertoire of abstract data types. In addition to defining the abstractions, we show how to implement them in C++ and we analyze the performance of the algorithms.

The repertoire of basic abstract data types has been designed as a hierarchy of C++ classes. This section presents an overview of the class hierarchy and lays the groundwork for the following chapters.

  • Class Hierarchy
  • Objects
  • The NullObject Singleton Class
  • Object Wrappers for the Built-In Types
  • Containers
  • Visitors
  • Iterators
  • The NullIterator Class
  • Direct vs. Indirect Containment
  • Ownership of Contained Objects
  • Associations
  • Searchable Containers


What This Book Is About

This book is about the fundamentals of data structures and algorithms--the basic elements from which large and complex software artifacts are built. To develop a solid understanding of a data structure requires three things: First, you must learn how the information is arranged in the memory of the computer. Second, you must become familiar with the algorithms for manipulating the information contained in the data structure. And third, you must understand the performance characteristics of the data structure so that when called upon to select a suitable data structure for a particular application, you are able to make an appropriate decision.

This book also illustrates object-oriented design and it promotes the use of common, object-oriented design patterns. The algorithms and data structures in the book are presented in the C++ programming language. Virtually all the data structures are presented in the context of a single class hierarchy. This commitment to a single design allows the programs presented in the later chapters to build upon the programs presented in the earlier chapters.


Class Hierarchy

The C++ class hierarchy which is used to represent the basic repertoire of abstract data types is shown in Figure gif . Two kinds of classes are shown in Figure gif ; abstract C++ classes , which look like this [image], and concrete C++ classes , which look like this [image]. Lines in the figure indicate derivation; base classes always appear to the left of derived classes.

figure4459
Figure: Object Class Hierarchy

An abstract class is a class which specifies an interface only. It is not possible create object instances of abstract classes. In C++ an abstract class typically has one or more pure virtual member functions . A pure virtual member function declares an interface only--there is no implementation defined. In effect, the interface specifies the set of operations without specifying the implementation.

An abstract class is intended to be used a the base class from which other classes are derived . Declaring the member functions virtual makes it possible to access the implementations provided by the derived classes through the base-class interface. Consequently, we don't need to know how a particular object instance is implemented, nor do we need to know of which derived class it is an instance.

This design pattern uses the idea of polymorphism . Polymorphism literally means ``having many forms.'' The essential idea is that a single, common abstraction is used to define the set of values and the set of operations--the abstract data type. This interface is embodied in the C++ abstract class definition. Then, various different implementations (many forms) of the abstract data type can be made. This is done in C++ by deriving concrete class instances from the abstract base class.

The remainder of this section presents the top levels of the class hierarchy which are shown in Figure gif . The top levels define those attributes of objects which are common to all of the classes in the hierarchy. The lower levels of the hierarchy (i.e., those derived from the Container class) are presented in subsequent chapters where the abstractions are defined and various implementations of those abstractions are elaborated.

figure4484
Figure: Object Class Hierarchy


Objects

The abstract class at the top of the class hierarchy is called Object. With the exception of the Ownership class, all the other classes in the hierarchy are derived from this class. Program gif gives the declaration of the Object class. Altogether only six member functions are declared: the destructor, IsNull, Hash, Put, Compare and CompareTo.

program4502
Program: Object Class Definition

The Object class destructor is declared virtual. It is essential to declare it so, because the Object is an abstract class, and instances of derived class objects are very likely to be accessed through the base class interface. In particular, it is necessary for the destructor of the derived class to be called through the base class interface--this is precisely what the virtual keyword achieves.

The IsNull function is a pure virtual member function which returns a Boolean value. This function is used in conjunction with the NullObject concrete classes described below in Section gif . The IsNull function shall return false for all object instances derived from Object except if the object instance is the NullObject. We will see later that certain functions return object references. For example, a function which searches through a data structure for a particular object returns a reference to that object if it is found. If the object is not found, the search returns a reference to the NullObject instance. By using the IsNull function, the programmer can test whether the search was successful. This is analogous to the NULL pointer in the C programming language and to the pointer value 0 in C++.

The Hash function is a pure virtual member function which returns a HashValue. The Hash function is used in the implementation of hash tables which are discussed in Chapter gif . We have put the Hash function in the Object class interface, so that it is possible to store any object derived from Object in a hash table. What the Hash function computes is unspecified. The only requirement is that it is idempotent . I.e., given an object instance obj, obj.Hash() does not modify the object in any way, and as long as obj is not modified in the interim, repeated calls always give exactly the same result.

Two functions for comparing objects are declared--Compare and CompareTo. Compare is a public member function. that takes a const reference to Object and returns an int. Given two objects obj1 and obj2, calling obj1.Compare (obj2) compares the value of obj1 with the value of obj2. The result is equal to zero if [image]; less than zero if [image]; and, greater than zero if [image].

It is not necessary that the two objects compared using Compare have the same type. As long as they are instances of classes derived from Object, they can be compared. The Compare function is used in the overloading of the comparison operators operator==, operator!=, operator<, operator<=, operator>, and operator>=, as shown in Program gif .

The second comparison function, CompareTo, is a pure virtual function. An implementation for this function must be given in every concrete class derived from Object. The purpose of this function is to compare two objects that are both instances of the same derived class.

The purpose of the Put member function of the Object class is to output a human-readable representation of the object on the specified output stream. The Put function is a pure virtual function. However, since it has the side-effect of output, it is not strictly idempotent. Nevertheless, it is a const member function which means that calling the Put member function does not modify the object in any way. The use of the Put in the overloading of the ostream insertion operator, operator<<, is shown in Program gif .

program4588
Program: Object Operator Definitions

The use of polymorphism in the way shown gives the programmer enormous leverage. The fact almost all objects will be derived from the Object base class, together with the fact that the Put and CompareTo member functions are virtual functions, ensures that the overloaded operators work as we expect for all derived class instances.

  • Implementation


Implementation

Only three functions need to be implemented to complete the definition of the Object class. These are shown in Program gif .

program4614
Program: Object Class Member Function Definitions

The Object class destructor is trivial. Since there are no member variables, nothing remains to be cleaned up. It is necessary to define the destructor in any event. As explained above, the destructor needs to be virtual and because in C++ there is no such thing as a pure virtual destructor.

The implementation of the IsNull member function is trivial. It simply returns the Boolean value false.

The Compare member function makes use of run-time type information to ensure that the CompareTo member function is only invoked for objects which are of the same class. For objects which are not instances of the same class, the before function is used to order those objects based on their types. The C++ run-time library provides an implementation for the before function which returns true if the left hand type precedes the right hand type in the implementation's collation order.


The NullObject Singleton Class

The declaration of the NullObject class is given in Program gif . The NullObject class is a concrete class derived from the Object abstract base class. The NullObject class is also a singleton class. I.e., it is a class of which there will only ever be exactly one instance. The one instance is in fact a static member variable of the class itself. In order to ensure that no other instances can be created, the default constructor is declared to be a private member function. The static member function Instance returns a reference to the one and only NullObject instance.

program4647
Program: NullObject Class Definition

  • Implementation


Implementation

Program gif shows how the member functions of the NullObject class are defined. The most important characteristic of the NullObject class is that its IsNull member function returns the value true.

program4672
Program: NullObject Class Member Function Definitions

The remaining functions are trivial: The constructor does nothing; the two member functions Hash and CompareTo both return zero; and the Put function simply prints ``NullObject''. Finally, the Instance static member function returns a reference to the static member variable instance which is the one and only instance of the NullObject class.


Object Wrappers for the Built-In Types

One of the design goals of the C++ language is to treat user-defined data types and the built-in data types as equally as possible. However, the built-in data types of C++ do have one shortcoming with respect to user-defined ones--a built-in data type cannot be used as the base class from which other classes are derived. E.g., it is not possible to write:

class D : public int // Wrong!
Consequently, it is not possible to extend the functionality of a built-in type using inheritance.

The usual design pattern for dealing with this deficiency is to put instances of the built-in types inside a wrapper class and to use the wrapped objects in place of the built-in types. Program gif illustrates this idea. The class Wrapper<T> is a concrete class which is derived from the abstract base class Object. It is also a generic class, the purpose of which is to encapsulate an object of type T. Each instance of the Wrapper<T> class contains a single member variable of type T called datum.

program4708
Program: Wrapper<T> Class Definition

  • Implementation


Implementation

The implementation of the Wrapper<T> class member functions is shown in Program gif . The Wrapper<T> class default constructor simply initializes the member variable datum using its default constructor. A second constructor takes as its lone argument a const reference to an object of type T and copies that value to the member variable datum. In effect, the constructor wraps its argument.

program4736
Program: Wrapper<T> Class Member Function Definitions

A type-cast operator is defined, operator T const&, which converts from an object of type Wrapper<T> to a const reference to an object of type T. In effect, the type-cast operator unwraps the contained object! This, together with the automatic type coercion rules of C++, makes it possible to use the wrapped object in the same context in which the unwrapped object is expected.

The Hash member function of the Wrapper<T> class simply calls the global (non-member) function, ::Hash(T). In effect, we have punted on the implementation of the Hash function. The implementation of a suitable hash function for an object of type T depends on the actual type used. However, the Wrapper<T> class is generic--we don't know what T is. Hash functions are discussed in Chapter gif .

The CompareTo function takes as its lone argument a const reference to Object. It uses the dynamiccast operator to convert the argument to the type Wrapper<T>. In C++, the dynamic cast will succeed only if the type of the referenced actually is Wrapper<T>. Otherwise a badcast exception is thrown. The CompareTo member function is implemented in the same way as the Hash member function. I.e., it calls the global (non-member) function ::Compare to do the actual comparison.

The implementation of the Put member function is trivial. It simply inserts datum into the given ostream using operator<<. Again, this assumes that the operator has already been defined for objects of type T.

Program gif uses the Wrapper<T> class defined above to define the Int, Char, Double, and String object classes which simply encapsulate variables of type int, char, double, and string, respectively.

program4794
Program: Int, Char, Double and String Class Definitions

These declarations require the existence of suitable ::Compare and ::Hash functions. E.g., the Int class definition requires the existence of a Compare(int) and a Hash(int) function. Hash functions are described in Chapter gif . The implementation of a suitable compare function is left as a project for the reader (Project gif ).


Containers

The Container class abstracts the notion of a container--an object that holds within it other objects. A container is itself an object. Therefore, the Container class is derived from the Object class. A consequence of this is that containers can be held in other containers!

The declaration of the Container class is given in Program gif . Notice that the Container declaration uses multiple inheritance . It is derived from two base classes, Object and Ownership. We have already seen the definition of the Object class. The Ownership class is defined below in Section gif .

program4829
Program: Container Class Definition

The Container class is an abstract base class. It is intended to be used as the base class from which concrete container realizations are derived. The Container class public interface comprises eight virtual member functions--Count, IsEmpty, IsFull, Hash, Put, NewIterator, Purge and Accept. Default implementations are provided for the first six; the last two are pure virtual member functions.

A single member variable, count, is declared. This variable is used to keep track of the number of objects held in the container. The count field is set initially to zero. It is the responsibility of the derived class to update this field as required.

The Count member function is an accessor that returns the number of items contained in the container. The IsEmpty and IsFull functions are Boolean-valued accessors which indicate whether a given container is empty or full, respectively. The Hash function is used in conjunction with hash tables as discussed in Chapter gif and the Put function inserts a human-readable representation of the container and its contents in a specified output stream. The purpose of the Purge function is to remove all the objects from a container, making it empty.

In order to describe the remaining two member functions, Accept and NewIterator, we need to become familiar first with the concepts of a visitor and an iterator, as well as with the Visitor and Iterator classes which embody these concepts. Visitors are discussed below in Section gif and iterators are discussed in Section gif .

Program gif gives code that defines the Container class constructor as well as the default behaviors for several of the member functions. The three functions Count, IsEmpty and IsFull are member variable accessors--they are idempotent const member functions which provide information about the status of the container.

program4884
Program: Container Class Member Function Definitions

The Count function simply returns the value of the count member variable. The IsEmpty function is a Boolean-valued accessor which returns the value true if the container is empty, i.e., it contains no objects. Notice that the IsEmpty function does not directly access the count field. Instead it calls Count which is virtual function.

In some cases, a container is implemented in a way which makes its capacity finite. When this is the case, it is necessary to be able to determine when the container is full. The IsFull function is a Boolean-valued accessor which returns the value true if the container is full. However, the default version always returns false.

Default behaviors are also provided by the Container class for the Hash and Put operations. The implementation of the Put function is discussed in the following section. The Hash function is given in Chapter gif .


Visitors

The Container class described in the preceding section interacts closely with the Visitor class shown in Program gif . In particular, the Accept member function of the Container class takes as its lone argument a reference to a Visitor.

program4922
Program: Visitor Class Definition

But what is a visitor? A shown in Program gif , a visitor is an object that has the two member functions Visit and IsDone. Of these, the Visit function is the most interesting. The Visit function takes as its lone argument a reference to an Object instance.

The interaction between a container and a visitor goes like this. The container is passed a reference to a visitor by calling the container's Accept member function. I.e., the container ``accepts'' the visitor. What does a container do with a visitor? It calls the Visit member function of that visitor one-by-one for each object contained in the container.

The interaction between a Container and its Visitor are best understood by considering an example. The following code fragment gives the design framework for the implementation of the Accept function in some concrete class, say SomeContainer, which is derived from the abstract base class Container:

void SomeContainer::Accept (Visitor& visitor) const

    for each Object i in this container

        visitor.Visit (i);

The Accept function calls Visit for each object i in the container. Since the class Visitor is an abstract base class which does not provide an implementation for the Visit operation, what the visitor actually does with an object depends on the type of visitor used.

Suppose that we want to print all of the objects in the container. One way to do this is to create a PrintingVisitor which prints every object it visits, and then to pass the visitor to the container by calling the Accept member function. The following code shows how we can declare the PrintingVisitor class which prints an object on the standard output stream, cout.

class PrintingVisitor : public Visitor
{
public:
    void Visit (object& object)
        { cout << object; }
};

Finally, given a container c that is an instance of a concrete container class SomeContainer which is derived from the abstract base class Container, we can call the Accept function as follows:

SomeContainer c;
PrintingVisitor v;
c.Accept (v);
The effect of this call is to call the Visit member function of the visitor for each object in the container.

  • The IsDone Member Function
  • Container Class Default Put Member Function


The IsDone Member Function

As shown in Program gif , the Visitor class interface also includes the member function IsDone. The IsDone member function is an accessor which is used to determine whether a visitor has finished its work. I.e., the IsDone member function returns the boolean value true if the visitor ``is done.''

The idea is this: Sometimes a visitor does not need to visit all the objects in a container. I.e., in some cases, the visitor may be finished its task after having visited only a some of the objects. The IsDone member function can be used by the container to terminate the Accept function like this:

void SomeContainer::Accept (Visitor& visitor) const

    for each Object i in this container

        if (visitor.IsDone ())
            return;
        visitor.Visit (i);

To illustrate the usefulness of IsDone, consider a visitor which visits the objects in a container with the purpose of finding the first object that matches a given object. Having found the first matching object in the container, the visitor is done and does not need to visit any more contained objects.

The following code fragment defines a visitor which finds the first object in the container that matches a given object.

class MatchingVisitor : public Visitor
{
    Object const& target;
    Object* found;
public:
    MatchingVisitor (Object const& object) :
        target (object), found (0)
        {}
    void Visit (Object& object)
    {
        if (found == 0 && object == target)
            found = &object;
    }
    bool IsDone ()
        { return found != 0; }
};

The constructor of the MatchingVisitor visitor takes a reference to an Object instance that is the target of the search. I.e., we wish to find an object in a container that matches the target. For each object the MatchingVisitor visitor visits, it compares that object with the target and makes found point at that object if it matches. Clearly, the MatchingVisitor visitor is done when the found pointer is non-zero.

Suppose we have a container c that is an instance of a concrete container class, SomeContainer, which is derived from the abstract base class Container; and an object x that is an instance of a concrete object class, SomeObject, which is derived from the abstract base class Object. Then, we can call use the MatchingVisitor visitor as follows:

SomeContainer c;
SomeObject x;
MatchingVisitor v (x);
c.Accept (v);

The observant reader will have noticed in Program gif that the Visit member function of the abstract Iterator class is pure virtual function whereas the IsDone function is not. It turns out that it is convenient to define a default implementation for the IsDone function that always returns false.


Object-Oriented Design

Traditional approaches to the design of software have been either data oriented or process oriented. Data-oriented methodologies emphasize the representation of information and the relationships between the parts of the whole. The actions which operate on the data are of less significance. On the other hand, process-oriented design methodologies emphasize the actions performed by a software artifact; the data are of lesser importance.

It is now commonly held that object-oriented methodologies are more effective for managing the complexity which arises in the design of large and complex software artifacts than either data-oriented or process-oriented methodologies. This is because data and processes are given equal importance. Objects are used to combine data with the procedures that operate on that data. The main advantage of using objects is that they provide both abstraction and encapsulation.

  • Abstraction
  • Encapsulation


Container Class Default Put Member Function

Program gif gives the definition of the default Put function for containers. Recall that the Put function is a virtual function inherited by the Container class from the abstract base class Object.

program5005
Program: Container Class Put Function Definition

A default Put function is provided to simplify the implementation of classes derived from the Container class. The default behavior is to print out the name of the class and then to print each of the elements in the container, by using the Accept member function with the visitor PuttingVisitor.

This implementation makes us of run-time type information to determine the name of the class. The C++ operator typeid returns a reference to an instance of the typeinfo class which is defined in the standard header file typeinfo.h . This class has member function, name() , which returns a pointer to a character string that contains the name of the class.


Iterators

In this section we introduce an abstraction called an iterator. An iterator provides a means for visiting one-by-one all the objects in a container. Iterators are an alternative to using the visitors described in Section gif . The basic idea is that for every concrete container class we will also implement a related concrete iterator derived from an abstract Iterator class.

Program gif gives the declaration of the abstract Iterator class. It defines an interface comprised of a virtual destructor and four pure virtual member functions--Reset, IsDone and two overloaded operators.

program5041
Program: Iterator Class Definition

The Iterator class is intended to be used as the base class from which other classes are derived in a polymorphic class hierarchy. Consequently, the destructor is declared as a virtual member function. Since the Iterator class is an abstract class which has no member variables, the behavior of the destructor trivial--it does nothing.

In addition to the destructor, the Iterator class interface comprises the Reset and IsDone functions and two overloaded operators--operator* and operator++. In order to understand the desired semantics, it is best to consider first an example which illustrates the use of an iterator.

Consider the implementation of a concrete container class, say SomeContainer, which is derived from the abstract base class Container. Associated with this container class is a concrete iterator, say SomeIterator, which is derived from the abstract base class Iterator. The following code fragment serves to illustrate the use of the iterator to visit one-by-one the objects contained in the container:

SomeContainer c;
Iterator& i = c.NewIterator ();
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
delete &i;

The NewIterator function of the SomeContainer class is defined as follows:

Iterator& SomeContainer::NewIterator () const
    { return *new SomeIterator (*this); }
I.e., given an instance c of SomeContainer, the call
c.NewIterator ();
results in the creation of a new instance of SomeIterator associated with container c.

In order to have the desired effect, the member functions IsDone, operator*, and operator++, must have the following behaviors:

IsDone
The IsDone member function is called in the loop-termination test of the while statement. The IsDone function returns false if the iterator still refers to an object in the container, and true when the container has been exhausted. I.e., if all of the contained objects have been visited.
operator*
The pointer dereferencing operator, operator*, is used to access the object to which the iterator currently refers. If this function is called when the container has been exhausted, a reference to the NullObject instance is returned.
operator++
The pre-increment operator is used to advance the iterator to the next object in the container. If the container is exhausted, the increment operator has no effect on the iterator.

Given these semantics for the iterator operators, the program fragment shown above systematically visits all of the objects in the container and prints each one on its own line of the standard output file.

After an iterator has exhausted all the contained objects, it can be reset via the Reset function and used again like this:

Iterator& i = c.NewIterator ();
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
i.Reset ()
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
delete &i;

One of the advantages of using an iterator object which is separate from the container is that it is possible then to have more than one iterator associated with a given container. This provides greater flexibility than possible using a visitor, since only one visitor can be accepted by the container at any given time. E.g., consider the following code fragment:

SomeContainer c;
Iterator& i = c.NewIterator ();
while (!i.IsDone ()) {
    Iterator& j = c.NewIterator ();
    while (!j.IsDone ()) {
        if (*i == *j)
            cout << *i << endl;
    ++j
    }
    delete &j;
    ++i;
}
delete &i;
This code compares all pairs of objects, (i,j), in the container and prints out those which are equal.

A certain amount of care is required when defining and using iterators. In order to simplify the implementation of iterators, we shall assume that while an iterator is in use, the associated container will not be modified. Specifically, this means that the no non-const member function of the associated container may be called. In particular, this also means that the container must not be deleted while an iterator is in use!


The NullIterator Class

A NullIterator is a concrete iterator that does nothing. As discussed in Section gif , every container has a NewIterator member function that returns a reference to an iterator instance. If a particular concrete container class does not provide an associated iterator class, its NewIterator member function returns a reference to a NullIterator. Program gif gives the declaration of the NullIterator class.

program5101
Program: NullIterator Class Definition

Because it is a concrete class, NullIterator provides implementations for all the pure virtual member functions of the Iterator class. As shown in Program gif , the implementation is trivial: The IsDone member function always returns true; the operator* member function always returns a reference to the NullObject instance; and the remaining functions do nothing at all.

program5125
Program: NullIterator Class Member Function Definitions

The default behavior of the NewIterator function of the Container class is to return a reference to a a new NullIterator instance as shown in Program gif .

program5145
Program: Container Class NewIterator Member Function Definition


Direct vs. Indirect Containment

The notion of a container as an object which contains other objects is introduced in Section gif . Conspicuous by their absence from the interface are member functions for putting objects into the container and for taking objects out of the container. As it turns out, the particular forms of the functions required depend on the type of container implemented. Therefore, we have left the specification of those functions for the classes which are derived from the Container base class.

Another reason for leaving these matters unspecified, is that we have not yet defined what it means for one object to be contained within another! We have two options:

Direct Containment
When an object is put into a container, a copy of that object is made in the container.
Indirect Containment
When an object is put into a container, a pointer to that object is kept in the container.

The main advantage of using direct containment is its simplicity. It is easy to understand and easy to implement. However, it does suffer some problems. First, if the objects which are to be put into a container are large, i.e., if they occupy a large amount of memory space, the copying of the objects is likely to be both time-consuming and space-consuming. Second, an object cannot be contained in more than one container at a time. Third, a container cannot contain itself.

The indirect containment approach addresses these three concerns. By keeping a pointer to the contained object rather than a copy of the contained object, it is not necessary to copy the object when it is put into the container--this saves time and space. By keeping pointers to the contained objects, it is possible to put a pointer to the same object into several different containers. Finally, it is possible to put the pointer to a container into the container itself.

The indirect containment approach is not without disadvantages. The principal disadvantage being that for every access to a contained object it is necessary to dereference a pointer. This adds a constant overhead to such accesses. Despite the disadvantages, the Container classes presented in the following chapters are all implemented using indirect containment.


Ownership of Contained Objects

A matter that is closely related to the containment of objects is the ownership of objects. Ownership is important because it is the owner of an object that ensures that the object's destructor is called and that any storage dynamically allocated for the object is freed when the object is no longer needed.

The ownership of contained objects is clear when direct containment is used. Since the container makes a copy of any object put into the container, it is the container that is responsible for deleting the copy when it is longer needed. In particular, the destructor for the container would normally delete all contained objects.

Unfortunately, the issue of ownership is not as clear when indirect containment is used. Indirect containment means that when an object is put into a container, it is a pointer to that object which is recorded in the container. This implies that the object was initially created outside of the container. The question then becomes, should the container delete the object when the time comes to clean up or should the deletion be done by outside of the container?

If we assume that it is the responsibility of the container to delete the contained objects, then we must make sure that the only objects put into a container are objects whose storage was dynamically allocated. This is because as the owner of the object, the container must delete the object when the time comes to clean up. But, given a pointer to an object, it is not possible for the container to know whether the object it points to has been dynamically allocated or whether it is actually a statically allocated global variable or a stack allocated local variable.

Another consequence of assigning the ownership of objects to a container is that things become complicated when an object is put into two or more different containers as well as when a given object is put into the same container more than once. The problem is that an object inserted in more than one container has more than one owner. In order to ensure that the object is only deleted once, we would have to extract the pointer from all but one of the containers before deleting them.

On the other hand, if we assume that it is not the responsibility of the of the container to delete contained objects, then the responsibility to clean up falls on the user of the container. In order to ensure that all contained objects are properly deleted, it is necessary to extract all of the contained objects from a container before deleting the container itself.

The solution to this dilemma is to support both paradigms. I.e., make it possible for the user of a container to specify whether that container is to be the owner of its objects. The Ownership class given in Program gif does precisely that.

program5170
Program: Ownership Class Definition

The Ownership class encapsulates a single Boolean variable, isOwner, which records whether the container is the owner of the contained objects. By default, the isOwner field is set to true in the constructor. Two member functions, AssertOwnership and RescindOwnership, provide a means for the user of a container to change the state of the isOwner datum. The IsOwner accessor reveals the current ownership status.

The behavior of the copy constructor is subtle: It transfers ownership status from the original container to the copy. This behavior is useful because it simplifies the task of returning a container as the result of a function.

In its declaration in Program gif we saw that the Container class was derived both from the Object base class and from the Ownership class. Therefore, all containers have ownership associated with them.

The following code fragment gives the design framework for the implementation of the Purge member function in some concrete class, say SomeContainer, which is derived from the abstract base class Container:

void SomeContainer::Purge ()

    if (IsOwner ())

        for each Object i in this container

            delete &i;

    Now clean up the container itself.
The container calls its own IsOwner member function to determine whether it is the owner of its contained objects. Then, and only if it is the owner, the contained objects are deleted. Given the Purge function, the implementation of the destructor is trivial:
SomeContainer::~SomeContainer ()
    { Purge (); }


Associations

An association is an ordered pair of objects. The first element of the pair is called the key ; the second element is the value associated with the given key.

Associations are useful for storing information in a database for later retrieval. E.g., a database can be viewed as a collection of key-and-value pairs. The information associated with a given key is retrieved from the database by searching the database for an the ordered pair in which the key matches the given key.

An association is like a container in that it contains two objects--the key and the value. On the other hand, an association is fundamentally different from a container since the contained objects are treated very differently. Normally, the objects in a container are not distinguished one from another. In an association, the two contained objects are distinct--one of them is the key and the other is a value associated with the key.

Program gif gives the declaration of the concrete object class Association. The declaration of the Association class is like that of Container class in that both of them are derived from the same set of base classes--Object and Ownership.

program5223
Program: Association Class Definition

An association has two member variables, key and value, both of which are of type pointer to Object. Since pointers are used, the Association class implements indirect containment.

  • Implementation


Implementation

Two constructors and the destructor are defined in Program gif . The first constructor takes as its lone argument a reference to an Object. It makes its key member variable point at the specified object and sets the value pointer to zero. The second constructor takes two arguments--both of them a reference to an Object. The key and value member variables are made to point at the corresponding objects.

program5258
Program: Association Class Constructor and Destructor

The destructor follows the general framework for the destructor of a class derived from Ownership. If the association is the owner of the contained key and value, then those objects must be deleted by the destructor.

The remaining member functions of the Association class are defined in Program gif . The first two, Key and Value, are member variable accessors. The former returns a reference to the key object obtained by dereferencing the key member variable; the latter, a reference to the value object obtained by dereferencing the value member variable.

program5275
Program: Association Class Member Functions

The CompareTo function is one place where an association distinguishes between the key and the value. The CompareTo function only operates on the key. The CompareTo function takes as its lone argument a const reference to Object. It uses the dynamiccast operator to convert the argument to an Association. The result of the comparison is based solely on the keys of the objects--the values have no rôle in the comparison.

Finally, the purpose of the Put member function is to print out a representation of the association. The implementation is trivial and needs no further explanation.


Searchable Containers

A searchable container is an extension of the container abstraction. It adds to the interface provided for containers functions for putting objects in and taking objects out, for testing whether a given object is in the container, and a routine to search the container for a given object.

The declaration of the SearchableContainer class is shown in Program gif . The SearchableContainer class is an abstract class derived from the abstract base class Container. It adds four more pure virtual member functions to the inherited abstract interface.

program5306
Program: SearchableContainer Class Definition

The IsMember function is a Boolean-valued function which takes as its lone argument a const reference to Object. The purpose of this routine is to test whether the given object instance is in the container. In effect, the routine searches the container for a pointer to the given object.

The purpose of the Insert member function is to put an object into the container. The Insert function takes as its argument a reference to Object. Remember, we have chosen to implement containers using indirect containment. In addition, it depends on the ownership status of the container whether it will delete the objects it contains.

The Withdraw function is used to remove an object from a container. The lone argument is a reference to the object to be removed. After an object has been removed from a container, the container no longer owns that object. In any event, the container never deletes an object which is removed from the container.

The final member function, Find, is used to locate an object in a container and to return a reference to that object. In this case, it is understood that the search is to be done using the object comparison functions. I.e., the routine is not to be implemented as a search of the container for a pointer to the given object but rather as a search of the container for an object which compares equal to the given object.

There is an important subtlety in the semantics of Find. The search is not for the given object, but rather for an object which compares equal to the given object. These semantics are particularly useful when using associations. Recall that two associations will compare equal if their keys compare equal--the values may be different.

In the event that the Find function fails to find an object equal to the specified object, then it will return a reference to the NullObject instance. Therefore, the user of the Find function should test whether the object which is returned is the NullObject instance by calling the IsNull() member function.

The Find function does not remove the object it finds from the container. Therefore, a container which owns its contained objects retains ownership of the result of the search. An explicit call of the Withdraw function is needed to actually remove the object from the container.


Exercises

  1. Specify the set of values and the set of operations provided by each of the following C++ built-in data types:
    1. char,
    2. int,
    3. double, and
    4. string.
  2. What are the features of C++ that facilitate the creation of user-defined data types.
  3. Explain how each of the following C++ features supports polymorphism:
    1. operator overloading,
    2. templates, and
    3. inheritance.
  4. Suppose we define two concrete classes, A and B, both of which are derived from the Object class declared in Program gif . Furthermore, let a and b be instances of classes A and B (respectively) declared as follows:
    class A : public Object { ... };
class B : public Object { ... };
A a;
B b;
    Give the sequence of functions called in order to evaluate a comparison such as ``a<b''. Is the result of the comparison true or false? Explain.
  5. Consider the Wrapper<T> class defined in Program gif . Explain the operation of the following program fragment:
    int i = 5;
Wrapper<int> j = 7;
i = j;
j = i;
  6. There are three ways to test whether a given object instance, obj, is the null object:
    1. obj.IsNull ()
    2. obj == NullObject::Instance (), and
    3. &obj == &(NullObject::Instance ()).
    Discuss the relative merits of each approach.
  7. Let c be an instance of some concrete class derived from the Container class given in Program gif . Explain how the statement
    cout << c;
    prints the contents of the container on the standard output stream, cout.
  8. Suppose we have a container c (i.e., an instance of some concrete class derived from the Container class defined in Program gif ) which among other things happens to contain itself. Is it permissible for c to own the objects it contains? What happens when c's destructor runs if it owns the objects it contains.
  9. Iterators and visitors provide two ways to do the same thing--to visit one-by-one all the objects in a container. Give an implementation for the Accept function of the Container class that uses an iterator.
  10. Is it possible to implement an iterator using a visitor? Explain.
  11. Suppose we have a container which we know contains only instances of the Int class defined in Program gif . Design a Visitor which computes the sum of all the integers in the container.
  12. Explain what the following visitor does and why it is a horribly bad idea:
    class DeletingVisitor : public Visitor
{
public:
    void Visit (Object& object)
        { delete &object; }
};
  13. Consider the following pair of Associations:
    Association a (*new Int (3), *new Int (4));
Association b (*new Int (3));
    Give the sequence of functions called in order to evaluate a comparison such as ``a==b''. Is the result of the comparison true or false? Explain.


Projects

  1. Design and implement suitable Compare functions for the C++ built-in types int, char, double and string so that they may be wrapped using the Wrapper<T> class declared in Program gif .
  2. Using visitors, devise implementations for the IsMember and Find member functions of the SearchableContainer class declared in Program gif .
  3. Using iterators, devise implementations for the IsMember and Find member functions of the SearchableContainer class declared in Program gif .
  4. Devise a scheme using visitors whereby all of the objects contained in one searchable container can be removed from it and transfered to another container.
  5. A bag is a simple container that can hold a collection of objects. Design and implement a concrete class called Bag derived from the SearchableContainer class declared in Program gif . Use the Array<T> class given in Chapter gif to keep track of the contents of the bag.
  6. Repeat Project gif , this time using the LinkedList<T> class given in Chapter gif .
  7. Ownership is all or nothing--a container owns all the object it contains or none of them. However, sometimes it is useful for an association to be the owner of the key but not of the associated value. Design and implement a class called Assoc which has this characteristic. Derive the class Assoc from the class Association given in Program gif .
  8. The Java programming language provides the notion of an enumeration as the means to iterate through the objects in a container. In C++ we can define enumerations like this:
    class Enumeration
{
public:
    virtual bool hasMoreElements () const = 0;
    virtual Object& nextElement () = 0;
};
    Given an enumeration e for some container c, the contents of c can be printed like this:
    while (e.hasMoreElements ())
    cout << e.nextElement () << endl;
    Devise a wrapper class to encapsulate an iterator and provide the functionality of an enumeration.


Abstraction

Abstraction can be thought of as a mechanism for suppressing irrelevant details while at the same time emphasizing relevant ones. An important benefit of abstraction is that it makes it easier for the programmer to think about the problem to be solved.

For example, procedural abstraction lets the software designer think about the actions to be performed without worrying about how those actions are implemented. Similarly, data abstraction lets the software designer think about the objects in a program and the interactions between those objects without having to worry about how those objects are implemented.

There are also many different levels of abstraction. The lower the levels of abstraction expose more of the details of an implementation whereas the higher levels hide more of the details.


Stacks, Queues and Deques

In this chapter we consider several related abstract data types--stacks, queues and deques. Each of these can be viewed as a pile of items. What distinguishes each of them is the way in which items are add to or removed from the pile.

In the case of a stack, items are added to and removed from the top of the pile. Consider the pile of papers on your desk. Suppose you add papers only to the top of the pile or remove them only from the top of the pile. At any point in time, the only paper that is visible is the one on top. What you have is a stack.

Now suppose your boss comes along and asks you to immediately complete a form. You stop doing whatever it is you are doing, and place the form on top of your pile of papers. When you have filled-out the form, you remove it from the top of the stack and return to the task you were working on before your boss interrupted you. This example illustrates that a stack can be used to keep track of partially completed tasks.

A queue is a pile in which items are added an one end and removed from the other. In this respect, a queue is like the line of customers waiting to be served by a bank teller. As customers arrive, they join the end of the queue while the teller serves the customer at the head of the queue. As a result, a queue is used when a sequence of activities must be done on a first-come, first-served basis.

Finally, a deque extends the notion of a queue. In a deque, items can be added to or removed from either end of the queue. In a sense, a deque is the more general abstraction of which the stack and the queue are just special cases.

  • Stacks
  • Queues
  • Deques
  • Exercises
  • Projects


Stacks

figure5583
Figure: Object Class Hierarchy

The simplest of all the containers is a stack . A stack is a container which provides exactly one function, Push, for putting objects into the container; and one function, Pop, for taking objects out of the container. Figure gif illustrates the basic idea.

figure5595
Figure: Basic Stack Operations

Objects which are stored in stack are kept in a pile. The last item put into the stack is a the top. When an item is pushed into a stack, it is placed at the top of the pile. When an item popped, it is always the top item which is removed. Since it is always the last item to be put into the stack that is the first item to be removed, a stack is a last-in, first-out or LIFO data structure.

In addition to the Push and Pop operations, the typical stack implementation also provides an accessor called Top which returns the item at the top of the stack without removing it from the stack.

Program gif gives the Stack abstract class definition. The Stack class is derived from the Container class. Hence, its interface comprises all of the member functions inherited from the base class plus the three member functions Push, Pop, and Top. Notice also that the Stack class is an abstract class. The functions Push, Pop, and Top are declared as pure virtual functions.

program5790
Program: Stack Class Definition

When implementing a data structure, the first issue to be addressed is to select the foundational data structure(s) to use. Often, the choice is between an array-based implementation and a pointer-based implementation. The next two sections show an array-based implementation of stacks which uses the Array<T> class introduced in Chapter gif and a pointer-based implementation which uses the LinkedList<T> class.

  • Array Implementation
  • Linked List Implementation
  • Applications


Array Implementation

This section describes an array-based implementation of stacks. Program gif gives the declaration of the StackAsArray class. The StackAsArray class is a concrete object class. Therefore, its definition provides an implementation for all of the inherited member functions which are pure in the base class definition. Most of the function prototypes have been elided from Program gif for the sake of brevity.

program5811
Program: StackAsArray Class Definition

  • Member Variables
  • Constructor and Destructor
  • Push, Pop, and Top Member Functions
  • The Accept Member Function
  • Iterator


Member Variables

The StackAsArray class definition contains one member variable--array. The variable array is declared by instantiating the Array<T> class template with [image]. I.e., array is an array of pointers to Objects. The use of pointers is in keeping with the design decision to implement containers using indirect storage.


Constructor and Destructor

The definitions of the StackAsArray class constructor and destructor are given in Program gif . The constructor takes a single parameter, size, which specifies the maximum number of items that can be stored in the stack. The variable array is initialized to be an array of length size. Its constructor requires O(1) time to construct an array of pointers. Therefore, the total running time for the StackAsArray constructor is O(1).

program5841
Program: StackAsArray Class Constructor, Destructor and Purge Member Function Definitions

The StackAsArray destructor simply calls the Purge member function. The behavior of the Purge function depends on whether the stack (as a container) is the owner of the contained objects. If it is the owner, then the contained objects must be deleted by the Purge function. In this case, the Purge function deletes one-by-one the first count elements of the array. In general, because of the polymorphic implementation of Objects, we cannot know the running time to delete the array elements. However, if we assume that the destructors for all the objects each run in constant time, the total running time for the Purge function is O(n), where [image], i.e., n is the number of elements in the stack.


Push, Pop, and Top Member Functions

Program gif defines the Push, Pop, and Top, member functions of the StackAsArray class. The first of these, Push, adds an element to the stack. It takes as its lone argument a reference to the Object to be pushed onto the stack.

program5873
Program: StackAsArray Class Push, Pop, and Top Member Function Definitions

The Push function first checks to see if there is room left in the stack. If no room is left, it throws a domainerror exception. Otherwise, it simply puts a pointer to the object into the array, and then increases the count variable by one. In a correctly functioning program, stack overflow should not occur. If we assume that overflow does not occur, the running time of the Push function is clearly O(1).

The Pop function removes an item from the stack and returns a reference to that item. If the stack is the owner of its contained objects, then when an item is removed from the stack, that item ceases to be owned by the stack. Consequently, the Pop function never invokes operator delete. The Pop function first checks if the stack is empty. If the stack is empty, it throws a domainerror exception. Otherwise, it simply decreases count by one and returns a reference to the item found at the top of the stack. In a correctly functioning program, stack underflow will not occur normally. The running time of the Pop function is O(1).

Finally, the Top function is a stack accessor which returns a const reference to the top item in the stack. The Top function is a const member function since it does not modify the stack. In particular, it does not remove the top item from the stack. The Top function first checks if the stack is empty. If the stack is empty, it throws a domainerror exception. Otherwise, it returns a reference to the top item, which is found at position [image] in the array. Assuming stack underflow does not occur normally, the running time of the Top function is O(1).


The Accept Member Function

Program gif defines the Accept member function for the StackAsArray class. As discussed in Chapter gif , the purpose of the Accept member function of a container, is to accept a visitor, and cause it to visit one-by-one all of the contained objects.

program5914
Program: StackAsArray Class Accept Member Function Definition

In the array implementation of the stack, the elements contained in the container occupy positions 0, 1, ..., [image] of the array. The body of the Accept function is simply a loop which calls the Visit function for each object in the stack. The running time of the Accept function depends on the running time of the function Visit. Let [image] be the running time of the function Visit. In addition to the time for the function call, each iteration of the loop incurs a constant overhead. Consequently, the total running time for Accept is [image], where n is the number of objects in the container. And if [image], the total running time is to O(n).


Iterator

In addition to the StackAsArray class, Program gif also defines the nested class Iter. Notice that the StackAsArray::Iter class is a friend of the StackAsArray class. The friend of a class has access to the private member variables and functions of that class. Consequently, the implementation of the iterator depends on the implementation of the container.

program5938
Program: StackAsArray::Iter Class Member Function Definitions

Recall from Chapter gif that an iterator is meant to be used like this:

StackAsArray stack;
stack.Push (*new Int (3));
stack.Push (*new Int (1));
stack.Push (*new Int (4));
Iterator& i = stack.NewIterator ();
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
delete &i;
This example declares the variable stack, pushes several Int objects onto the stack, and then uses an iterator to systematically print out all of the elements contained in the stack.

The StackAsArray::Iter class definition is given in Program gif . Two member variables are defined--stack and position. The former is a const reference to a StackAsArray; the latter, an unsigned integer. The StackAsArray::Iter constructor takes as its lone argument a const reference to an StackAsArray object and it makes the stack member variable refer to that object. Then, it calls the Reset member function which sets the position member variable to zero. The effect of all this is to associate the iterator with the given stack instance and to make it refer to the first element (i.e., the one at the bottom) of the associated stack. Clearly, the running time of the StackAsArray::Iter constructor is O(1).

The IsDone member function is called in the loop termination test of the while loop given above. The purpose of IsDone member function is to determine when all of the contained objects have been exhausted. In Program gif this occurs when the variable position is equal to the count variable of the associated stack. The running time of IsDone is O(1).

The dereferencing operator, operator*, is called in the body of the for loops to access the object to which the iterator refers. It returns a reference to the appropriate Object in the stack, provided that the list has not been exhausted, i.e., provided that the value of the position variable in the range between 0 and [image]. Otherwise, it returns a reference to the NullObject instance when the position is invalid.

Finally, the increment operator, operator++, is used to cause the iterator to advance its position to the next contained object. In this case, advancing the position simply means adding one. This operator does nothing if the position is initially invalid. In either case, the running time is simply O(1).


Linked List Implementation

In this section we will examine a pointer-based implementation of stacks that makes use of the LinkedList<T> data structure developed in Chapter gif . Program gif gives the declaration of the concrete object class StackAsLinkedList and its associated iterator, StackAsLinkedList::Iter. As in the preceding section, the function prototypes for the interface inherited from the Container base class have been elided for the sake of brevity. Of course, implementations must be provided for all of the inherited member functions which are declared pure in the base class.

program5997
Program: StackAsLinkedList Class Definition

  • Member Variables
  • Constructor and Destructor
  • Push, Pop, and Top Member Functions
  • The Accept Member Function
  • Iterator


Member Variables

The implementation of the StackAsLinkedList class makes use of one member variable--list. The variable list is an instance of the LinkedList<T> class template with [image]. Thus, list is a linked list of pointers to Objects. This list will be used to keep track of the elements in the stack. Since we have decided to implement containers using indirect storage, the linked list contains pointers to the objects in the stack.


Encapsulation

Encapsulation aids the software designer by enforcing information hiding. Objects encapsulate data and the procedures for manipulating that data. In a sense, the object hides the details of the implementation from the user of that object.

There are two very real benefits from encapsulation--conceptual and physical independence. Conceptual independence results from hiding the implementation of an object from the user of that object. Consequently, the user is prevented from doing anything with an object that depends on the implementation of that object. This is desirable because it allows the implementation to be changed without requiring the modification of the user's code.

Physical independence arises from the fact that the behavior of an object is determined by the object itself. The behavior of an object is not determined by some external entity. As a result, when we perform an operation on an object, there are no unwanted side-effects.


Constructor and Destructor

The definitions of the constructor and destructor functions for StackAsLinkedList objects are given in Program gif . In the case of the linked list implementation, it is not necessary to preallocate storage. When using a linked list, the storage is allocated dynamically and incrementally on the basis of demand.

program6024
Program: StackAsLinkedList Class Constructor, Destructor and Purge Member Function Definitions

The constructor simply initializes the list variable using the default constructor for the LinkedList<T> class. The LinkedList<T> constructor runs in constant time. Thus, the running time of the StackAsLinkedList constructor is O(1).

The StackAsLinkedList destructor simply calls the Purge member function. The Purge function deletes the contained objects if it owns them. Hence, the Purge function goes through the elements of the linked list, and follows the pointers to delete the objects one-by-one. In general, because of the polymorphic implementation of Objects, we cannot know the running time to delete an object. However, if we assume that the destructors for all the objects each run in constant time, the total running time for the Purge function is O(n), where [image]. I.e., n is the number of elements in the stack.


Push, Pop, and Top Member Functions

The Push, Pop, and Top, member functions of the StackAsLinkedList class are defined in Program gif .

program6056
Program: StackAsLinkedList Class Push, Pop, and Top Member Function Definitions

The implementation of Push is trivial. It takes as its lone argument a reference to the Object to be pushed onto the stack and simply prepends a pointer to that object to the linked list list. Then, one is added to the count variable. The running time of the Push function is constant, since the Prepend function has a constant running time, and updating the count only takes O(1) time.

The Pop function is implemented using two of the LinkedList<T> member functions--First and Extract. The First function is used to obtain the first item in the linked list. The function First runs in constant time. The Extract function is then called to remove the first item from the linked list. In the worst case, Extract requires O(n) time to delete an item from a linked list of length n. But the worst-case time arises only when it is the last element of the list which is to be deleted. In the case of the Pop function, it is the first element which is deleted. This can be done in constant time. Assuming that the exception which is raised when Pop is called on an empty list does not occur, the running time for Pop is O(1).

There is a subtle point in this implementation: Since it is not possible for two different variables to occupy the same memory address, gif all of the pointers in the linked list will be unique provided no object is pushed onto the stack more than once. This ensures that the datum deleted from the linked list by the Extract function is precisely the one returned by the First function.

The only way that an object can be pushed safely onto the stack more than once is if the stack does not own its contained objects. If the stack owns its objects, then when the stack is deleted its destructor first deletes all the contained objects. As a result, if an object is pushed onto the stack twice, that object's destructor would be called twice. Such a program is not valid since deleting an object which has already been deleted is an error.

The definition of the Top function is quite simple. It simply returns a reference to the first object in the linked list. Provided the linked list is not empty, the running time of Top is O(1). If the linked list is empty, the Top function throws a domainerror exception.


The Accept Member Function

The Accept member function of the StackAsLinkedList class is defined in Program gif . The Accept member function takes a visitor and calls its Visit function one-by-one for all of the objects on the stack.

program6106
Program: StackAsLinkedList Class Accept Member Function Definition

The implementation of the Accept function for the StackAsLinkedList class mirrors that of the StackAsArray class shown in Program gif . In this case, the linked list is traversed from front to back, i.e., from the top of the stack to the bottom. As each element of the linked list is encountered, the Visit function is called. If [image] is the running time of the function Visit, the total running time for Accept is [image], where [image] is the number of objects in the container. If we assume that [image], the total running time is O(n).


Iterator

The StackAsLinkedList::Iter class is declared in Program gif . Since the StackAsLinkedList::Iter class is declared as a friend of the StackAsLinkedList class the member functions of the former can access the private member variables of the latter. Consequently, the implementation of the iterator depends on the implementation of the container.

program6130
Program: StackAsLinkedList::Iter Class Member Function Definitions

StackAsLinkedList::Iter objects have two member variables, stack and position. The former is a reference to a StackAsLinkedList instance. The latter is declared as a ListElement<Object*> const*, i.e, a pointer to a const element of a linked list of pointers to Objects.

The StackAsLinkedList::Iter constructor simply calls the Reset member function. The Reset function makes position point at the first element of the linked list which represents the stack, by calling the Head function of the LinkedList<T> class. Clearly the running time of the constructor is O(1).

The IsDone member function simply tests for the null pointer. If the position variable is zero, IsDone returns true. Again, the running time is clearly O(1).

The dereferencing operator, operator*, does what its name says! It returns a const reference to the object obtained by dereferencing the object pointer contained in the list element to which the position variable points. This takes O(1) time. Note that if pointer is zero, a reference to the NullObject instance is returned.

Finally, the increment operator, operator++, advances the position pointer to the next element of the linked list. This is done by calling the Next function of the ListElement<T> class. This too is accomplished in constant time.

Because the interface of the StackAsLinkedList class is exactly the same as that of the StackAsArray class, StackAsLinkedList objects can be used in exactly the same way as their array-based counterparts:

StackAsLinkedList stack;
stack.Push (*new Int (3));
stack.Push (*new Int (1));
stack.Push (*new Int (4));
Iterator& i = stack.NewIterator ();
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
delete &i;
This program fragment declares the variable stack, pushes several values onto the stack, and then uses an iterator to systematically print out all of the elements contained in the stack.


Applications

Consider the following expression:

In order to determine the value of this expression, we first compute the sum 5+9 and then multiply that by 2. Then we compute the product [image] and add it to the previous result to get the final answer. Notice that the order in which the operations are to be done is crucial. Clearly if the operations are not done in the correct order, the wrong result is computed.

The expression above is written using the usual mathematical notation. This notation is called infix notation. What distinguishes this notation is the way that expressions involving binary operators are written. A binary operator is an operator which has exactly two operands, such as + and [image]. In infix notation, binary operators appear in between their operands.

Another characteristic of infix notation is that the order of operations is determined by operator precedence . For example, the [image] (multiplication) operator has higher precedence than does the + (addition) operator. When an evaluation order is desired that is different from that provided by the precedence, parentheses , ``('' and ``)'', are used to override precedence rules. I.e., an expression in parentheses is evaluated first.

As an alternative to infix, the Polish logician Jan [image] ukasiewicz introduced notations which require neither parentheses nor operator precedence rules. The first of these, the so-called Polish notation , places the binary operators before their operands. I.e., for Equation gif we would write:

This is also called prefix notation, because the operators are written in front of their operands.

While prefix notation is completely unambiguous in the absence of parentheses, it is not very easy to read. A minor syntactic variation on prefix is to write the operands as a comma-separated list enclosed in parentheses as follows:

While this notation seems somewhat foreign, in fact it is precisely the notation that is used for function calls in C++:

operator+ (operator* (operator+ (5,9) ,2), operator* (6,5));

The second form of [image] ukasiewicz notation is the so-called Reverse-Polish notation (RPN ). Equation gif is written as follows in RPN:

This notation is also called postfix notation for the obvious reason--the operators are written after their operands.

Postfix notation, like prefix notation, does not make use of operator precedence nor does it require the use of parentheses. A postfix expression can always be written without parentheses that expresses the desired evaluation order. E.g., the expression [image], in which the multiplication is done first, is written [image]; whereas the expression [image] is written [image].

  • Evaluating Postfix Expressions
  • Implementation


Evaluating Postfix Expressions

One of the most useful characteristics of a postfix expression is that the value of such an expression can be computed easily with the aid of a stack of values. The components of a postfix expression are processed from left to right as follows:

  1. If the next component of the expression is an operand, the value of the component is pushed onto the stack.
  2. If the next component of the expression is an operator, then its operands are in the stack. The required number of operands are popped from the stack; the specified operation is performed; and the result is pushed back onto the stack.
After all the components of the expression have been processed in this fashion, the stack will contain a single result which is the final value of the expression. Figure gif illustrates the use of a stack to evaluate the RPN expression given in Equation gif .

figure6232
Figure: Evaluating the RPN Expression in Equation gif using a Stack


Implementation

Program gif gives the implementation of a simple RPN calculator. The purpose of this example is to illustrate the use of the Stack class. The program shown accepts very simplified RPN expressions: The expression may contain only single-digit integers, the addition operator, +, and the multiplication operator, *. In addition, the operator = pops the top value off the stack and prints it on the standard output stream, cout. Furthermore, the calculator does its computation entirely with integers.

program6591
Program: Stack Application--A Single-Digit, RPN Calculator

Notice that the RPNCalculator function is passed a reference to a Stack object. Consequently, the function manipulates the stack entirely through the abstract interface. The calculator does not depend on the stack implementation used! For example, if we wish to use a stack implemented using an array, we would declare a StackAsArray variable and invoke the calculator as follows:

StackAsArray s (10);
RPNCalculator (s);
On the other hand, if we decided to use the pointer-based stack implementation, we would write:
StackAsLinkedList s;
RPNCalculator (s);

The running time of the RPNCalculator function depends upon the number of symbols, operators and operands, in the expression being evaluated. If there are n symbols, the body of the for loop is executed n times. It should be fairly obvious that the amount of work done per symbol is constant, regardless of the type of symbol encountered. This is the case for both the StackAsArray and the StackAsLinkedList stack implementations. Therefore, the total running time needed to evaluate an expression comprised of n symbols is O(n).


Queues

In the preceding section we saw that a stack comprises a pile of objects that can be accessed only at one end--the top. In this section we examine a similar data structure called a single-ended queue . Whereas in a stack we add and remove elements at the same end of the pile, in a single-ended queue we add elements at one end and remove them from the other. Since it is always the first item to be put into the queue that is the first item to be removed, a queue is a first-in, first-out or FIFO data structure. Figure gif illustrates the basic queue operations.

figure6612
Figure: Basic Queue Operations

Program gif gives the Queue abstract class definition. The Queue class is derived from the Container class. The Queue class interface comprises all the functions inherited from the base classes plus the three function, Head, Enqueue, and Dequeue.

program6890
Program: Queue Class Definition

As we did with stacks, we will examine two queue implementations--an array-based one and a pointer-based one. The array-based implementation uses the Array<T> class and the pointer-based implementation, the LinkedList<T> class, both of which are defined in Chapter gif .

  • Array Implementation
  • Linked List Implementation
  • Applications


Array Implementation

Program gif declares the QueueAsArray class. The QueueAsArray class is a concrete class that provides an array-based queue implementation. As such, it must provide an implementation for all of the inherited pure virtual functions--the function prototypes have been elided for the sake of brevity.

program6910
Program: QueueAsArray Class Definition

  • Member Variables
  • Constructor and Destructor
  • Head, Enqueue, and Dequeue Member Functions


Member Variables

QueueAsArray objects comprise three member variables--array, head and tail. In keeping with the decision to use indirect storage in the implementation of containers, the variable array is declared as an array of pointers to Objects.

The pointers to the objects contained in the queue will be held in a contiguous range of array elements as shown in Figure gif (a). The variables head and tail denote the left and right ends, respectively, of this range. In general, the region of contiguous elements will not necessarily occupy the leftmost array positions. As elements are deleted at the head, the position of the left end will change. Similarly, as elements are added at the tail, the position of the right end will change. In some circumstances, the contiguous region of elements will wrap around the ends of the array as shown in Figure gif (b).

figure6932
Figure: Array Implementation of a Queue

As shown in Figure gif , the leftmost element is array[head], and the rightmost element is array[tail]. When the queue contains only one element, [image] as shown in Figure gif (c).

Finally, Figure gif (b) shows that if the queue is empty, the head position will actually be to the right of the tail position. However, this is also the situation which arises when the queue is completely full! The problem is essentially this: Given an array of length n, then [image] and [image]. Therefore, the difference between the head and tail satisfies [image]. Since there are only n distinct differences, there can be only n distinct queue lengths, 0, 1, ..., n-1. It is not possible to distinguish the queue which is empty from the queue which has n elements solely on the basis of the head and tail member variables.

There are two options for dealing with this problem: The first is to limit the number of elements in the queue to be at most n-1. The other is to use another member variable, count, to keep track explicitly of the actual number of elements in the queue rather than to infer the number from the head and tail variables. The latter approach has been adopted in the implementation given below.


Object Hierarchies and Design Patterns

There is more to object-oriented programming than simply encapsulating in an object some data and the procedures for manipulating those data. Object-oriented methods deal also with the classification of objects and they address the relationships between different classes of objects.

The primary facility for expressing relationships between classes of objects is derivation--new classes can be derived from existing classes. What makes derivation so useful is the notion of inheritance. Derived classes inherit the characteristics of the classes from which they are derived. In addition, inherited functionality can be overridden and additional functionality can be defined in a derived class.

A feature of this book is that virtually all the data structures are presented in the context of a single class hierarchy. In effect, the class hierarchy is a taxonomy of data structures. Different implementations of a given abstract data structure are all derived from the same abstract base class. Related base classes are in turn derived from classes that abstract and encapsulate the common features of those classes.

In addition to dealing with hierarchically related classes, experienced object-oriented designers also consider very carefully the interactions between unrelated classes. With experience, a good designer discovers the recurring patterns of interactions between objects. By learning to use these patterns, your object-oriented designs will become more flexible and reusable.

Recently, programmers have to started name the common design patterns. In addition, catalogs of the common patterns are now being compiled and published[14].

The following object-oriented design patterns are used throughout this text:

  • Containers
  • Iterators
  • Visitors
  • Adapters
  • Singletons


Constructor and Destructor

The definitions of the QueueAsArray class constructor and destructor are given in Program gif . The constructor takes a single parameter, size, which specifies the maximum number of items that can be stored in the queue. The constructor initializes the member variables as follows: The variable array is initialized to an array of length size and the remaining variables, head and tail, are initialized to represent the empty queue. Therefore, the total running time for the QueueAsArray constructor is O(1).

program7244
Program: QueueAsArray Class Constructor, Destructor and Purge Member Function Definitions

The QueueAsArray destructor simply calls the Purge member function. The Purge function is responsible for deleting the contained objects if the queue (as a container) is the owner of those objects. If the queue owns the contained objects, the Purge function deletes them all one-by-one, starting from at the head and moving toward the tail. Because we cannot know the types of the objects contained, we cannot know the running time required to delete each of them. However, if we assume that the destructors for all the objects each run in constant time, the total running time for the QueueAsArray class Purge function is O(n), where [image], i.e., n is the number of items in the queue.


Head, Enqueue, and Dequeue Member Functions

Program gif defines the Head, Enqueue and Dequeue member functions of the QueueAsArray class.

program7272
Program: QueueAsArray Class Head, Enqueue and Dequeue Member Function Definitions

The Head member function simply returns a reference to the object found at the head of the queue, having first checked to see that the queue is not empty. If the queue is empty, it throws a domainerror exception. Under normal circumstances, we expect that the queue will not be empty. Therefore, the normal running time of this function is O(1).

The Enqueue function takes a single argument which is a reference to an object to be added to the tail of the queue. The Enqueue function first checks that the queue is not full--a domainerror exception is thrown when the queue is full. Next, the position at which to insert the new element is determined by increasing the member variable tail by one modulo the length of the array. Finally, a pointer to the object to be enqueued is put into the array at the correct position and the count is adjusted accordingly. Under normal circumstances (i.e., when the exception is not thrown), the running time of Enqueue is O(1).

The Dequeue function removes an object from the head of the queue and returns a reference to that object. First, it checks that the queue is not empty and throws an exception when it is. If the queue is not empty, the function sets aside a reference to the object at the head in the local variable result; it increases the head member variable by one modulo the length of the array; adjusts the count accordingly; and returns result. All this can be done in a constant amount of time so the running time of Dequeue is a constant.


Linked List Implementation

This section presents a queue implementation which makes use of the singly-linked list data structure, LinkedList<T>, that is defined in Chapter gif . Program gif declares the QueueAsLinkedList object class.

program7306
Program: QueueAsLinkedList Class Definition

  • Member Variables
  • Constructor and Destructor
  • Head, Enqueue and Dequeue Member Functions


Member Variables

Just like the StackAsLinkedList class, the implementation of the QueueAsLinkedList class requires only one member variable--list. The variable list is a linked list of pointers to Objects. It is used to keep track of the elements in the queue.


Constructor and Destructor

Program gif defines the QueueAsLinkedList constructor and destructor functions. In the case of the linked list implementation, it is not necessary to preallocate storage. The constructor simply initializes the list object as an empty list. The running time of the constructor is O(1).

program7327
Program: QueueAsLinkedList Class Constructor, Destructor and Purge Member Function Definitions

The QueueAsLinkedList destructor simply calls the Purge member function The Purge function deletes the contained objects one-by-one by traversing the linked list. Assuming that the running time of the destructor for each the contained objects is a constant, the total running time for the QueueAsLinkedList destructor is O(n), where [image] is the number of items in the queue.


Head, Enqueue and Dequeue Member Functions

The Head, Enqueue and Dequeue member functions of the QueueAsLinkedList class are given in Program gif .

program7353
Program: QueueAsLinkedList Class Enqueue, Head and Dequeue Member Function Definitions

The Head member function returns a const reference to the object at the head of the queue. The head of the queue is in the first element of the linked list. In Chapter gif we saw that the running time of LinkedList<T>::First is a constant, Therefore, the normal running time for the Head function is O(1).

The Enqueue function takes a single argument--a reference to the object to be added to the tail of the queue. The function simply calls the LinkedList<T>::Append function. Since the running time for Append is O(1), the running time of Enqueue is also O(1).

The Dequeue function removes an object from the head of the queue and returns a reference to that object. First, it verifies that the queue is not empty and throws an exception when it is. If the queue is not empty, Dequeue saves a reference to the first item in the linked list in the local variable result. Then the item of the linked list is removed from the list. When using the LinkedList<T> class from Chapter gif , the time required to delete the first item from a list is O(1) regardless of the number of items in the list. As a result, the running time of Dequeue is also O(1).


Applications

The FIFO nature of queues makes them useful in certain algorithms. E.g., we will see in Chapter gif that a queue is an essential data structure for many different graph algorithms. In this section we illustrate the use of a queue in the breadth-first traversal of a tree.

Figure gif shows an example of a tree. A tree is comprised of nodes (indicated by the circles) and edges (shown as arrows between nodes). We say that the edges point from the parent node to a child node. The degree of a node is equal to the number of children of that node. E.g., node A in Figure gif has degree three and its children are nodes B, C, and D. A child and all of its descendents is called a subtree .

figure7402
Figure: A Tree

One way to represent such a tree is to use a collection of linked structures. Consider the following class definition which is an abridged version of the NaryTree class described in Chapter gif .

class NaryTree : public Object
{
    Object* key;
    unsigned int const degree;
    Array<NaryTree*> subtree;
public:
    Object& Key () const;
    unsigned int Degree () const;
    NaryTree& Subtree (unsigned int) const;
};

Each NaryTree object represents one node in a tree. The member variable degree keeps track of the degree of the node and the variable subtree is an array of pointers to the children of the node. The key field points to an object which represents the contents of the node. E.g. in Figure gif , each node carries a one-character label. The key field is intended to be used to represent that label. The member functions Key, Degree, and Subtree provide read-only access to the corresponding private member variables.

One of the essential operations on a tree is a tree traversal . A traversal visits one-by-one all the nodes in a given tree. To visit a node means to perform some computation using the information contained in that node--e.g., print the key. The standard tree traversals are discussed in Chapter gif . In this section we consider a traversal which is based on the levels of the nodes in the tree.

Each node in a tree has an associated level which arises from the position of that node in the tree. E.g., node A in Figure gif is at level 0, nodes B, C, and D are at level 1, etc. A breadth-first traversal visits the nodes of a tree in the order of their levels. At each level, the nodes are visited from left to right. For this reason, it is sometimes also called a level-order traversal . The breadth-first traversal of the tree in Figure gif visits the nodes from A to L in alphabetical order.

One way to implement a breadth-first traversal of a tree is to make use of a queue as follows: To begin the traversal, the root node of the tree is enqueued. Then, we repeat the following steps until the queue is empty:

  1. Dequeue and visit the first node in the queue.
  2. Enqueue its children in order from left to right.
Figure gif illustrates the breadth-first traversal algorithm by showing the contents of the queue immediately prior to each iteration.

figure7697
Figure: Queue Contents during the Breadth-First Traversal of the Tree in Figure gif

  • Implementation


Implementation

Program gif defines the function BreadthFirstTraversal. This function takes as its lone argument a reference to an NaryTree which is the root of the tree to be traversed. The algorithm makes use of the QueueAsLinkedList data structure, which was defined in the preceding section, to hold the appropriate tree nodes.

The running time of the BreadthFirstTraversal algorithm depends on the number of nodes in the tree which is being traversed. Each node of the tree is enqueued exactly once--this requires a constant amount of work. Furthermore, in each iteration of the loop, each node is dequeued exactly once--again a constant amount of work. As a result, the running time of the BreadthFirstTraversal algorithm is O(n) where n is the number of nodes in the traversed tree.

program8010
Program: Queue Application--Breadth-First Tree Traversal


Deques

In the preceding section we saw that a queue comprises a pile of objects into which we insert items at one and and from which we remove items at the other end. In this section we examine an extension of the queue which provides a means to insert and remove items at both ends of the pile. This data structure is a deque . The word deque is an acronym derived from double-ended queue . gif

Figure gif illustrates the basic deque operations. A deque provides three operations which access the head of the queue, Head, EnqueueHead and DequeueHead, and three operations to access the tail of the queue, Tail, EnqueueTail and DequeueTail.

figure8031
Figure: Basic Deque Operations

Program gif gives the declaration of the Deque abstract class. Because a deque is an extension of the notion of a single-ended queue to a double ended queue, it makes sense for the Deque class to be derived from the Queue class.

program8381
Program: Deque Class Definition

Notice that the Deque class interface includes the Enqueue and Dequeue operations inherited from the Queue base class. In the base class only one enqueue is required because items are always enqueued at the tail and only one dequeue operation is required because items are always dequeued at the head. However, in a deque items can be enqueued and dequeued at either end.

In order to ensure consistent semantics, the Deque class provides the default behaviors for the Enqueue and Dequeue functions as shown in Program gif . Viz., the Enqueue function simply calls EnqueueTail and the Dequeue function calls DequeueHead.

program8416
Program: Deque Class Enqueue and Dequeue Member Function Definitions

Why have we chosen to derive the Deque class from the Queue class and not the other way around? When we have two abstractions, one of which is essentially a subset of the other, there are two possible implementation approaches:

Specialization
The more general abstraction is the base class and the restricted abstraction is the derived class. E.g., when using specialization we would derive the class Queue from the class Deque thus:
class Queue : public Deque { ... };
The Queue class interface should restrict access to only those base class member functions that are appropriate.
Generalization
The more restricted abstraction is the base class from which the more general abstraction is derived. E.g., when using generalization we would derive the class Deque from the class Queue thus:
class Deque : public Queue { ... };
The Deque class inherits and generalizes the interface of the Queue class.

Often when using generalization, it turns out that the inherited member functions need to be overridden because their functionality needs to be enhanced in some way. In those cases, specialization may be the preferred approach, since only one implementation needs to be written. The more general implementation serves the needs of both the general base class and the specialized derived class.

On the other hand, making the base class more general and the derived class more specialized means that sometimes we have more functionality at our disposal than we really need. If we only have single-ended queues, we don't want the overhead associated with double-ended queue operations. For this reason, we have chosen the generalization approach.

  • Array Implementation
  • Linked List Implementation
  • Doubly-Linked and Circular Lists


Array Implementation

Program gif defines an array implementation of a deque. The DequeAsArray class is defined using multiple inheritance: It is derived from both the Deque abstract base class and the QueueAsArray array-based implementation of a queue. The Deque class provides the interface; the QueueAsArray, the implementation.

program8448
Program: DequeAsArray Class Definition

The QueueAsArray implementation provides almost all of the required functionality. For example, Program gif shows that the Head, EnqueueTail and DequeueHead operations of the DequeAsArray class are implemented by calling the appropriate QueueAsArray class operations.

program8464
Program: DequeAsArray Class Head EnqueueHead and DequeueHead Member Function Definitions

  • Tail, EnqueueHead, and DequeueTail Member Functions


Containers

A container is an object that holds within it other objects. A container has a capacity, it can be full or empty, and objects can be inserted and withdrawn from a container. In addition, a searchable container is a container that supports efficient search operations.


Tail, EnqueueHead, and DequeueTail Member Functions

Program gif defines the Tail, EnqueueHead and DequeueTail member functions of the DequeAsArray class.

program8487
Program: DequeAsArray Class Tail EnqueueTail and DequeueTail Member Function Definitions

The Tail member function simply returns a reference to the object found at the tail of the deque, having first checked to see that the deque is not empty. If the deque is empty, it throws a domainerror exception. Under normal circumstances, we expect that the deque will not be empty. Therefore, the normal running time of this function is O(1).

The EnqueueHead function takes a single argument which is a reference to an object to be added to the head of the deque. The EnqueueHead function first checks that the deque is not full--a domainerror exception is thrown when the deque is full. Next, the position at which to insert the new element is determined by decreasing the member variable head by one modulo the length of the array. Finally, a pointer to the object to be enqueued is put into the array at the correct position and the count is adjusted accordingly. Under normal circumstances (i.e., when the exception is not thrown), the running time of EnqueueHead is O(1).

The DequeueTail function removes an object from the tail of the deque and returns a reference to that object. First, it checks that the deque is not empty and throws an exception when it is. If the deque is not empty, the function sets aside a reference to the object at the tail in the local variable result; it decreases the tail member variable by one modulo the length of the array; adjusts the count accordingly; and returns result. All this can be done in a constant amount of time so the running time of DequeueTail is a constant.


Linked List Implementation

Program gif defines a linked-list implementation of a deque. The DequeAsLinkedList class is defined using multiple inheritance: It is derived from both the Deque abstract base class and the QueueAsLinkedList linked-list implementation of a queue. The Deque class provides the interface; the QueueAsLinkedList, the implementation.

program8523
Program: DequeAsLinkedList Class Definition

The QueueAsLinkedList implementation provides almost all of the required functionality. For example, Program gif shows that the Head, EnqueueTail and DequeueHead operations of the DequeAsLinkedList class are implemented by calling the appropriate QueueAsLinkedList class operations.

program8539
Program: DequeAsLinkedList Class Head, EnqueueHead and DequeueHead Member Function Definitions

  • Tail, EnqueueHead, and DequeueTail Member Functions


Tail, EnqueueHead, and DequeueTail Member Functions

Program gif defines the Tail, EnqueueHead and DequeueTail member functions of the DequeAsArray class.

program8562
Program: DequeAsLinkedList Class Tail, EnqueueTail and DequeueTail Member Function Definitions

The Tail member function returns a const reference to the object at the tail of the deque. The tail of the deque is in the last element of the linked list. In Chapter gif we saw that the running time of LinkedList<T>::Last is a constant, Therefore, the normal running time for the Tail function is O(1).

The EnqueueHead function takes a single argument--a reference to the object to be added to the head of the deque. The function simply calls the LinkedList<T>::Prepend function. Since the running time for Prepend is O(1), the running time of EnqueueHead is also O(1).

The DequeueTail function removes an object from the tail of the deque and returns a reference to that object. First, it verifies that the deque is not empty and throws an exception when it is. If the deque is not empty, DequeueTail saves a reference to the last item in the linked list in the local variable result. Then the item of the linked list is removed from the list. When using the LinkedList<T> class from Chapter gif , the time required to delete the last item from a list is O(n), where [image] is the number of items in the list. As a result, the running time of DequeueTail is O(n).


Doubly-Linked and Circular Lists

In the preceding section we saw that the running time of DequeueHead is O(1), but that the running time of DequeueTail is O(n), for the pointer based implementation of a deque. This is because the linked list data structure used, LinkedList<T> is a singly-linked list . Each element in a singly-linked list contains a single pointer--a pointer to the successor (next) element of the list. As a result, deleting the head of the linked list is easy: The new head is the successor of the old head.

However, deleting the tail of a linked list is not so easy: The new tail is the predecessor of the original tail. Since there is no pointer from the original tail to its predecessor, the predecessor must be found by traversing the linked list from the head. This traversal gives rise to the O(n) running time.

In a doubly-linked list , each list element contains two pointers--one to its successor and one to its predecessor. There are many different variations of doubly-linked lists: Figure gif illustrates four of them.

figure8602
Figure: Doubly-Linked and Circular List Variations

Figure gif (a) shows the simplest case: Two pointers, say head and tail, are used to keep track of the list elements. One of them points to the first element of the list, the other points to the last. The first element of the list has no predecessor, therefore that pointer is null. Similarly, the last element has no successor and the corresponding pointer is also null. In effect, we have two overlapping singly-linked lists which go in opposite directions. Figure gif also shows the representation of an empty list. In this case the head and tail pointers are both null.

Figure gif (b) shows a case which uses sentinels. In this variation two sentinels are used! Because there are in effect two overlapping linked lists that go in opposite directions, two sentinels are used--one for each singly-linked list. Recall, the use of a sentinel is motivated by the fact that the code for insertion and deletion is often simpler to write because there are fewer special cases to consider. Figure gif (b) shows that in the empty list, the two sentinels point to each other.

A circular, doubly-linked list is shown in Figure gif (c). A circular list is formed by making use of pointers which would otherwise be null: The last element of the list is made the predecessor of the first element; the first element, the successor of the last. The upshot is that we no longer need both a head and tail pointer to keep track of the list. Even if only a single pointer is used, both the first and the last list elements can be found in constant time.

Finally, Figure gif (d) shows a circular, doubly-linked list which has a single sentinel. This variation is similar to the preceding one in that both the first and the last list elements can be found in constant time. This variation has the advantage that no special cases are required when dealing with an empty list. Figure gif shows that the empty list is represented by a list with exactly one element--the sentinel. In the case of the empty list, the sentinel is both is own successor and predecessor. Since the sentinel is always present, and since it always has both a successor and a predecessor, the code for adding elements to the empty list is identical to that for adding elements to a non-empty list.


Exercises

  1. The array-based stack implementation shown in Programs gif , gif , gif and gif uses a fixed length array. As a result, it is possible for the stack to become full. However, the Array<T> class defined in Chapter gif provides a SetLength member function which can be used to change the length of the array.
    1. Rewrite the Push routine so that it doubles the length of the array when the array is full.
    2. Rewrite the Pop routine so that it halves the length of the array when the array is less than half full.
    3. Show that the average time for both push and pop operations is O(1). Hint: Consider the running time required to push [image] items onto an empty stack, where [image].
  2. Consider a sequence S of push and pop operations performed on a stack that is initially empty. The sequence S is a valid sequence of operations if at no point is a pop operation attempted on an empty stack and if the stack is empty at the end of the sequence. Design a set of rules for generating a valid sequence.
  3. Devise an implementation of the Queue abstract data type using two stacks. Give algorithms for the Enqueue and Dequeue operations, and derive tight big-oh expressions for the running times of your implementation.
  4. Write each of the following infix expressions in postfix notation:
    1. ,
    2. ,
    3. ,
    4. ,
    5. , and
    6. .
  5. Write each of the following postfix expressions in infix notation:
    1. ,
    2. ,
    3. ,
    4. ,
    5. , and
    6. .
  6. Devise an algorithm which translates a postfix expression to a prefix expression. Hint: Use a stack of strings.
  7. The array-based queue implementation shown in Programs gif , gif and gif uses a fixed length array. As a result, it is possible for the queue to become full.
    1. Rewrite the Enqueue routine so that it doubles the length of the array when the array is full.
    2. Rewrite the Dequeue routine so that it halves the length of the array when the array is less than half full.
    3. Show that the average time for both enqueue and dequeue operations is O(1).
  8. Stacks and queues can be viewed as special cases of deques. Show how all the operations on stacks and queues can be mapped to operations on a deque. Discuss the merits of using a deque to implement a stack or a queue.
  9. Suppose we add a new operation to the stack ADT called FindMinimum that returns a reference to the smallest element in the stack. Show that is its possible to provide an implementation for FindMinimum that has a worst case running time of O(1).
  10. The breadth-first traversal routine shown in Program gif visits the nodes of a tree in the order of their levels in the tree. Modify the algorithm so that the nodes are visited in reverse. Hint: Use a stack.


Projects

  1. Enhance the functionality of the RPN calculator given in Program gif in the following ways:
    1. Use double-precision, floating-point arithmetic. I.e., use the Double class defined in Program gif .
    2. Provide the complete repertoire of basic operators: +, -, [image] and [image].
    3. Add an exponentiation operator and a unary negation operator.
    4. Add a clear function that empties the operand stack and a print function that prints out the contents of the operand stack.
  2. Modify Program gif so that it accepts expressions written in prefix (Polish) notation. Hint: See Exercise gif .
  3. Write a program to convert a postfix expression into an infix expression using a stack. One way to do this is to modify the RPN calculator program given in Program gif to use a stack of infix expressions. The expressions can be represented as instances of the String class defined in Program gif . A binary operator should pop two strings from the stack and then push a string which is formed by concatenating the operator and its operands in the correct order. For example, suppose the operator is '*' and the two strings popped from the stack are "(b+c)" and "a". Then the result that gets pushed onto the stack is the string "a*(b+c)".
  4. Devise a scheme using a stack to convert an infix expression to a postfix expression. Hint: In a postfix expression operators appear after their operands whereas in an infix expression they appear between their operands. Process the symbols in the prefix expression one-by-one. Output operands immediately, but save the operators in a stack until they are needed. Pay special attention to the precedence of the operators.
  5. Modify your solution to Project gif so that it immediately evaluates the infix expression. I.e., create an InfixCalculator routine in the style of Program gif .
  6. Consider a string of characters, S, comprised only of the characters (, ), [, ], and . We say that S is balanced if it has one of the following forms:
    • , i.e., S is the string of length zero,
    • ,
    • ,
    • ,
    where both T and U are balanced strings, In other words, for every left parenthesis, bracket or brace, there is a corresponding right parenthesis, bracket or brace. E.g., "{()[()]}" is balanced, but "([)]" is not. Write a program that uses a stack of characters to test whether a given string is balanced. (Use the Char class defined in Program gif ).
  7. Design and implement a MultipleStack class which provides [image] stacks in a single container. The declaration of the class should look something like this:
    class MultipleStack : public Container
{
    // ...
public:
    MultipleStack (unsigned int);
    void Push (Object&, unsigned int);
    Object& Pop (unsigned int);
    // ...
};
    • The constructor takes a single integer argument that specifies the number of stacks in the container.
    • The Push function takes two arguments. The first gives the object to be pushed and the second specifies the stack on which to push it.
    • The Pop function takes a single integer argument which specifies the stack to pop.
    Choose one of the following implementation approaches:
    1. Keep all the stack elements in a single array.
    2. Use an array of Stack objects.
    3. Use a linked list of Stack objects.
  8. Design and implement a class called DequeAsDoublyLinkedList that provides a deque implemented as a doubly-linked list. Select one of the approaches shown in Figure gif .
  9. In Section gif , the Deque class is derived from the Queue class. This is the design paradigm known as generalization. The alternative paradigm is specialization in which the Queue class is derived from the Deque class. Redesign the Deque and Queue components of the class hierarchy using specialization.

    figure9062
    Figure: Expression Tree for

  10. Devise an approach for evaluating an arithmetic expression using a queue (rather than a stack). Hint: Transform the expression into a tree as shown in Figure gif and then do a breadth-first traversal of the tree in reverse (see Exercise gif ). E.g., the expression [image] becomes [image]. Evaluate the resulting sequence from left to right using a queue in the same way that a postfix expression is evaluated using a stack.


Ordered Lists and Sorted Lists

The most simple, yet one of the most versatile containers is the list . In this chapter we consider lists as abstract data types. A list is a series of items. In general, we can insert and remove items from a list and we can visit all the items in a list in the order in which they appear.

In this chapter we consider two kinds of lists--ordered lists and sorted lists. In an ordered list the order of the items is significant. Consider a list of the titles of the chapters in this book. The order of the items in the list corresponds to the order in which they appear in the book. However, since the chapter titles are not sorted alphabetically, we cannot consider the list to be sorted. Since it is possible to change the order of the chapters in book, we must be able to do the same with the items of the list. As a result, we may insert an item into an ordered list at any position.

On the other hand, a sorted list is one in which the order of the items is defined by some collating sequence. For example, the index of this book is a sorted list. The items in the index are sorted alphabetically. When an item is inserted into a sorted list, it must be inserted at the correct position.

The list abstractions can be implemented in many ways. In this chapter we examine implementations based on the array and the linked list foundational data structures presented in Chapter gif .

  • Ordered Lists
  • Sorted Lists
  • Exercises
  • Projects


Ordered Lists

figure9308
Figure: Object Class Hierarchy

The most basic of the searchable containers is an ordered list . In Chapter gif we defined a searchable container as a container which supports the following additional operations:

Insert
used to put objects into a the container;
Withdraw
used to remove objects from the container;
Find
used to locate objects in the container; and,
IsMember
used to test whether a given object instance is in the container.

An ordered list is a container which holds a sequence of objects. Each object has a unique position in the sequence. In addition to the basic repertoire of operations supported by all searchable containers, ordered lists provide the following operations:

FindPosition
used to find the position of an object in the ordered list;
operator []
used to access the object at a given position in the ordered list;
Withdraw(Position&)
used to remove the object at a given position from the ordered list.
InsertAfter
used to insert an object into the ordered list after the object at a given position; and
InsertBefore
used to insert an object into the ordered list before the object at a given position.

Program gif declares two abstract classes--List and OrderedList. The List class is derived from the SearchableContainer class which is in turn derived from the Container class. Consequently, the List class interface comprises all of the member functions inherited from these base classes plus four additional member functions, FindPosition, two versions of operator[] and Withdraw. As befits the definition of an abstract class, all of these functions are pure virtual member functions of the List class.

program9348
Program: List and Ordered Class Definitions

The OrderedList class extends the List class by adding two more member functions--InsertAfter and InsertBefore. The two functions provided by the OrderedList class have been separated out from the List class interface because the List class is used as the base class from which other types of lists are derived.

Program gif also defines the abstract class Position. The Position class abstracts the notion of the position of an item in a list. Since this is abstraction is almost identical to that of an iterator, the Position class is derived from the Iterator abstract class. No additional member functions are defined.

As we did in the previous chapter with stacks, deques and queues, we will examine two ordered list implementations--an array-based one and a pointer-based one. Section gif presents an implementation based on the Array<T> class; Section gif , a pointer-based implementation based on the LinkedList<T> class.

  • Array Implementation
  • Linked List Implementation
  • Performance Comparison: ListAsArray vs. ListAsLinkedList
  • Applications


Array Implementation

This section presents an array-based implementation of ordered lists. Program gif declares the ListAsArray class. The ListAsArray class is a concrete object class and as such must provide implementations for all of the interface member functions which have been declared as pure virtual functions in the base classes from which the ListAsArray class is derived. The prototypes have been omitted from Program gif since they have all been given earlier.

program9390
Program: ListAsArray Class Definition

  • Member Variables
  • Inserting and Accessing Items in a List
  • Finding Items in a List
  • Removing Items from a List
  • Positions of Items in a List
  • Finding the Position of an Item and Accessing by Position
  • Inserting an Item at an Arbitrary Position
  • Removing Arbitrary Items by Position


Member Variables

The ListAsArray class comprises one member variable, array, which is an array of pointers to Objects. The array variable is used to hold the items in the ordered list. Specifically, the items in the list are stored in array positions 0, 1, ..., [image]. In an ordered list the position of an item is significant. The item at position 0 is the first item in the list; the item at position [image], the last.

An item at position i+1 is the successor of the one at position i. I.e., the one at i+1 follows or comes after the one at i. Similarly, an item a position i is the predecessor of the one at position i+1; the one at position i is said to precede or to come before the one at i+1.


Iterators

An iterator provides a means by which the objects within a container can be accessed one-at-a-time. All iterators share a common interface, and hide the underlying implementation of the container from the user of that container.


Inserting and Accessing Items in a List

Program gif gives the definitions of the constructor and Insert member functions of the ListAsArray class. The constructor takes a single argument which specifies the length of array to use in the representation of the ordered list. Thus if we use an array-based implementation, we need to know when a list is declared what will be the maximum number of items in that list. The constructor initializes the array variable as an array with the specified length. The running time of the constructor is clearly O(1).

program9419
Program: ListAsArray Class Constructor, Insert Member Function and Subscripting Operator Definitions

The Insert member function is part of the interface of all searchable containers. I.e., it is a pure virtual function declared in the SearchableContainer class. Its purpose is to put an object into the container. The obvious question which arises is, where should the inserted item be placed in the ordered list? The simple answer is, at the end.

In Program gif we see that the Insert function simply adds the new item to the end of the list, provided there is still room in the array. Normally, the array will not be full, so the running time of this function is O(1).

Program gif also gives the definition of the subscripting operator, operator[], which take a subscript of type unsigned int. This function simply returns a reference to the object in the ordered list at the specified position. In this case, because the type unsigned int is used, the position is specified using a non-negative, integer-valued subscript expression. The implementation of this function is trivial--it simply makes use of the subscript operator provided by the Array<T> class. Assuming the specified position is valid, the running time of this function is O(1).


Finding Items in a List

Program gif defines two ListAsArray class accessor functions which search for an object in the ordered list. The IsMember function tests whether a particular object instance is in the ordered list. The Find function locates in the list an object which matches its argument.

program9447
Program: ListAsArray Class IsMember and Find Member Function Definitions

The IsMember function is a Boolean-valued function which takes as its lone argument a const reference to an Object. This function compares one-by-one the pointers contained in array with the address of the argument. Thus, this function tests whether a particular object instance is contained in the ordered list. In the worst case, the object sought is not in the list. In this case, the running time of the function is O(n), where [image] is the number of items in the ordered list.

The Find function also does a search of the ordered list. It also takes a single argument which is a const reference to an Object. However, find does not compare addresses. Instead, it uses operator== to compare the items. Thus, the Find function searches the list for an object which compares equal to its argument. The Find function returns a reference to the object found. If no match is found, it returns a reference to the NullObject instance. The running time of this function depends on the time required for the comparison operator, [image]. In the worst case, the object sought is not in the list. In this case the running time is [image]. For simplicity, we will assume that the comparison takes a constant amount of time. Hence, the running time of the function is also O(n), where [image] is the number of items in the list.

It is important to understand the subtle distinction between the search done by the IsMember function and and that done by Find. The IsMember function searches for a specific object instance while Find simply looks for a matching object. Consider the following:

Object& object1 = *new Int (57);
Object& object2 = *new Int (57);
ListAsArray list (1);
list.Insert (object1);
This code fragment creates two Int class object instances, both of which have the value 57. Only the first object, object1, is inserted into the ordered list list. Consequently, the function call
list.IsMember (object1)
returns true; whereas the function call
list.IsMember (object2)
returns false.

On the other hand, if a search is done using the Find function like this:

Object& object3 = list.Find (object2);
the search will be successful! After the call, object3 refers to object1.


Removing Items from a List

Objects are removed from a searchable container using the Withdraw function. Program gif defines the Withdraw function for the ListAsArray class. This function takes a single argument which is a reference to the object to be removed from the container. It is the specific object instance which is removed from the container, not simply one which matches (i.e., compares equal to) the argument.

program9491
Program: ListAsArray Class Withdraw Member Function Definition

The withdraw function first needs to find the position of the item to be removed from the list. This part is identical to the main loop of the IsMember function. An exception is thrown if the list is empty, or if the object to be removed is not in the list. The number of iterations needed to find an object depends on its position. If the object to be removed is found at position i, then the search phase takes O(i) time.

Removing an object from position i of an ordered list which is stored in an array requires that all of the objects at positions i+1, i+2, ..., [image], be moved one position to the left. Altogether, [image] objects need to be moved. Hence, this phase takes [image] time.

The running time of the Withdraw function is the sum of the running times of the two phases, O(i)+ [image]. Hence, the total running time is O(n), where [image] is the number of items in the ordered list.

Care must be taken when using the Withdraw function. Consider the following:

Object& object1 = *new Int (57);
Object& object2 = *new Int (57);
ListAsArray list (1);
list.Insert (object1);
To remove object1 from the ordered list, we may write
list.Withdraw (object1);
However, the call
list.Withdraw (object2);
will fail because object2 is not actually in the list. If for some reason we have lost track of object1, we can always write:
list.Withdraw (list.Find (object2));
which first locates the object in the ordered list (object1) which matches object2 and then deletes that object.


Positions of Items in a List

As shown in Program gif , the abstract class Position is derived from the class Iterator. Recall from Chapter gif that an iterator is used to systematically visit one-by-one all of the items in a container. An essential characteristic of an iterator is that at any instant, it refers to exactly one item in the container. To refer to an item in the container, it must keep track of the position of the item. Therefore, we may view an iterator as a kind of abstract position in an ordered list.

Program gif gives the declaration of the ListAsArray::Pos class which is a Position (and therefore an Iterator) associated with the ListAsArray ordered list class. ListAsArray::Pos objects contain two member variables--list and offset. The former is a reference to an ordered list; the latter records an offset in the corresponding array.

program9527
Program: ListAsArray::Pos Class Definition


Finding the Position of an Item and Accessing by Position

Program gif defines two more member functions of the ListAsArray class, FindPosition and operator[]. The FindPosition member function takes as its lone argument a const reference to an Object. The purpose of this function is to search the ordered list for an item which matches the object, and to return its position. The result is a reference to a Position. Since the Position class is an abstract base class, there can be no object instances of that class. However, there can be object instances of a concrete class derived from the Position class, such as the ListAsArray::Pos class. Therefore, the FindPosition function allocates an new instance of the ListAsArray::Pos class an returns a reference to that instance.

program9549
Program: ListAsArray Class FindPosition Member Function and Subscripting Operator Definitions

The search algorithm used in FindPosition is identical to that used in the Find routine (Program gif ). The FindPosition makes use of operator== to locate a contained object which is equal to the search target. Notice that if no match is found, the offset is set to the value count, which is one position to the right of the last item in the ordered list. The running time of FindPosition is identical to that of Find: , where [image].

The subscripting operator, operator[], defined in Program gif takes a const reference to a Position and returns a reference to the item in the ordered list at the given position. The Position argument is dynamically cast to an ListAsArray::Pos. Remember that in C++, run-time checks are made to ensure that the cast is safe. After a simple validity check, the offset recorded in the position is used to index into the array variable, to obtain the desired result. If the offset is equal to count, the position is invalid. In this case, a reference to the NullObject instance is returned. The running time of this operator is clearly O(1).


Inserting an Item at an Arbitrary Position

Two member functions for inserting an item at an arbitrary position in an ordered list are declared in Program gif --InsertBefore and InsertAfter. Both of these take two arguments: a const reference to a Position and a reference to an Object. The effects of these two functions are illustrated in Figure gif .

figure9589
Figure: Inserting an Item in an Ordered List Implemented as an Array

Figure gif shows that in both cases a number of items to the right of the insertion point need to be moved over to make room for the item that is being inserted into the ordered list. In the case of InsertBefore, items to the right including the item at the point of insertion are moved; for InsertAfter, only items to the right of the point of insertion are moved, and the new item is inserted in the array location following the insertion point.

Program gif gives the implementation of the InsertAfter member function for the ListAsArray class. The code for the InsertBefore function is identical except for one line as explained below.

program9799
Program: ListAsArray Class InsertAfter Member Function Definition

The InsertAfter function takes two arguments--a const reference to a Position and a reference to an Object. As was done in the FindPosition function, the first argument is dynamically cast to an ListAsArray::Pos. Next, some simple tests are done to ensure that the position is valid, and that there is room left in the array to do the insertion.

On line 11 the array index where the new item will ultimately be stored is computed. For InsertAfter the index is [image] as shown in Program gif . In the case of InsertBefore, the value required is simply offset. The loop on lines 13-14 moves items over and then a pointer object is saved on line 15.

If we assume that no exceptions are thrown, the running time of InsertAfter is dominated by the loop which moves list items. In the worst case, all the items in the array need to be moved. Thus, the running time of both the InsertAfter and InsertBefore functions is O(n), where [image].


Removing Arbitrary Items by Position

The final member function of the ListAsArray class that we will consider is the Withdraw function which takes a const reference to a Position. The desired effect of this function is to remove from the ordered list the item at the specified position.

Figure gif shows the way in which to delete an item from an ordered list which implemented with an array. All of the items remaining in the list to the right of the deleted item need to be shifted to the left in the array by one position.

figure9830
Figure: Withdrawing an Item from an Ordered List Implemented as an Array

Program gif gives the implementation of the Withdraw member function. As has been the pattern in all of the member functions which take a const reference to a Position, that argument is first dynamically cast to a ListAsArray::Pos. After checking the validity of the position, all of the items following the item to be withdraw are moved one position to the left in the array.

program10096
Program: ListAsArray Class Withdraw Member Function Definition

The running time of the Withdraw function depends on the position in the array of the item being deleted and on the number of items in the ordered lists. In the worst case, the item to be deleted is in the first position. In this case, the work required to move the remaining items left is O(n), where [image].


Linked List Implementation

This section presents a pointer-based implementation of ordered lists. Program gif declares the ListAsLinkedList class. gif Like its array-based counterpart ListAsArray, the ListAsLinkedList class is a concrete class derived from the abstract base class List. The sets of member functions comprising the public interfaces of the ListAsArray and ListAsLinkedList classes are identical.

program10118
Program: ListAsLinkedList Class Definition

  • Member Variables
  • Inserting and Accessing Items in a List
  • Finding Items in a List
  • Removing Items from a List
  • Positions of Items in a List
  • Finding the Position of an Item and Accessing by Position
  • Inserting an Item at an Arbitrary Position
  • Removing Arbitrary Items by Position


Member Variables

Objects of the ListAsLinkedList class contain one member variable, linkedList, which is a linked list of pointers to Objects. The linkedList is used to hold the items in the ordered list. Since a linked list is used, there is no notion of an inherent limit on the number of items which can be placed in the ordered list. Items can be inserted until the available memory is exhausted.


Inserting and Accessing Items in a List

Program gif gives the implementation of the Insert member function of the ListAsLinkedList class. This function takes a reference to an Object which is to be added to the ordered list. As in the case of the ArrayAsLinkedList class, the object is added at the end of the ordered list. This is done simply by calling the Append function from the LinkedList<T> class.

program10140
Program: ListAsLinkedList Class Constructor, Insert Member Function and Subscripting Operator Definitions

The running time of the Insert function is determined by that of Append. In Chapter gif this was shown to be O(1). The only other work done by the Insert function is to add one to the count variable. Consequently, the total running time for Insert is O(1).

Program gif also defines a subscripting operator, operator[], which takes an argument of type unsigned int. This operator is used to access elements of the ordered list by their position in the list. In this case, the position is specified by a non-negative, integer-valued subscript expression. Since there is no way to access directly the [image] element of linked list, the implementation of this function comprises a loop which traverses the list to find the [image] item. The function returns a reference to the [image] item, provided [image]. Otherwise, k is not a valid subscript value and the function throws an exception.

The running time of this operator[] depends on the number of items in the list and on the value of the subscript expression. In the worst case, the item sought is at the end of the ordered list. Therefore, the worst-case running time of this algorithm, assuming the subscript expression is valid, is O(n), where [image].


Visitors

A visitor represents an operation to be performed on all the objects within a container. All visitors share a common interface, and thereby hide the operation to be performed from the container. At the same time, visitors are defined separately from containers. Thus, a particular visitor can be used with any container.


Finding Items in a List

Program gif defines the IsMember and Find member functions of the ListAsLinkedList class. The implementations of these functions are almost identical. However, they differ in two key aspects--the comparison used and the return value.

program10173
Program: ListAsLinkedList Class IsMember and Find Member Function Definitions

The IsMember function tests whether a particular object instance is contained in the ordered list. It returns a Boolean value indicating whether the object is present. The running time of this function is clearly O(n), where [image], the number of items in the ordered list.

The Find member function locates an object which matches a given object. The match is determined by using operator==. Find returns a reference to the matching object if one is found. Otherwise, it returns a reference to the NullObject instance. The running time for this function, is [image], where [image] is the time required to do the comparison, and [image] is the number of items in the ordered list. This simplifies to O(n) when the comparison can be done in constant time.


Removing Items from a List

The Withdraw member function is used to remove a specific object instance from an ordered list. The implementation of the Withdraw member function for the ListAsLinkedList class is given in Program gif .

program10197
Program: ListAsLinkedList Class Withdraw Member Function Definition

The implementation of Withdraw is straight-forward: It simply calls the Extract function provided by the LinkedList<T> class to remove the specified object from the linkedList member variable. The running time of the Withdraw function is dominated by that of Extract which was shown in Chapter gif to be O(n), where n is the number of items in the linked list.


Positions of Items in a List

Program gif gives the definition of a the ListAsLinkedList::Pos class, the purpose of which is to record the position of an item in an ordered list implemented as a linked list.

program10216
Program: ListAsLinkedList::Pos Class Definition

The ListAsLinkedList::Pos class has two member variables--list and element. The first is a reference to the ordered list which contains the item of interest; the second is a pointer to the linked-list element in which the item appears. Notice that the implementation of the ListAsLinkedList::Pos class is fundamentally different from the array version, ListAsArray::Pos. In the array version, the position was specified by an offset, i.e, by an ordinal number that shows the position of the item in the ordered sequence. In the linked-list version, the position is specified by a pointer to the element of the linked list in which the item is stored. Regardless of the implementation, both kinds of position provide exactly the same functionality--their public interfaces are identical.


Finding the Position of an Item and Accessing by Position

The FindPosition member function of the ListAsLinkedList class is used to determine the position of an item in an ordered list implemented as a linked list. Its result is a reference to a Position. Such a position can be used with the subscripting operator, operator[] to access the item in the ordered list found at the given position. The FindPosition and operator[] member functions are defined in Program gif

program10239
Program: ListAsLinkedList Class FindPosition Member Function and Subscripting Operator Definitions

The FindPosition function takes as its lone argument a const reference to an Object that is the target of the search. The search algorithm used by FindPosition is identical to that of Find, which is given in Program gif . Consequently, the running time is the same:

where [image] is the time required to match to Objects, and [image] is the number of items in the ordered list.

The subscripting operator, operator[], which takes as its subscript expression a reference to a Position is also defined in Program gif . This routine assumes that the position it is passed is an instance of a ListAsLinkedList::Pos. It simply dereferences the element pointer contained in the ListAsLinkedList::Pos argument to obtain the required item in the ordered list. Assuming no exceptions are thrown, the running time is clearly O(1).


Inserting an Item at an Arbitrary Position

Once having determined the position of an item in an ordered list, we can make use of that position to insert items into the middle of the list. Two functions are specifically provided for this purpose--InsertAfter and InsertBefore. Both of these take the same set of arguments: A const reference to a Position which specifies the point at which the insertion is to be made, and a reference to the object to be inserted.

program10269
Program: ListAsLinkedList Class InsertAfter Member Function Definition

Program gif gives the implementation for the InsertAfter member function of the ListAsLinkedList class. After casting the arguments to the appropriate types and performing some validity checks, this function simply calls the InsertAfter function provided by the LinkedList<T> class. Assuming no exceptions are thrown, the running time for this function is O(1).

The implementation of InsertBefore is not shown--its similarity with InsertAfter should be obvious. Since it must call the InsertBefore routine provided by the LinkedList<T> class, we expect the worst case running time to be O(n), where [image].


Removing Arbitrary Items by Position

The final ListAsLinkedList member function to be considered is the version of Withdraw that takes as its lone argument a reference to a Position. The function removes an arbitrary item from an ordered list, where the position of that item is specified by an instance of ListAsLinkedList::Pos. The code for the Withdraw function is given in Program gif .

program10295
Program: ListAsLinkedList Class Withdraw Member Function Definition

The Withdraw function first converts its Position argument to the appropriate type--a ListAsLinkedList::Pos. It then performs validity checks on that position, checking that the position refers to this list, and that the position is valid. The item at the specified position in the list is removed from the linked list by calling the Extract function provided by LinkedList<T>.

The running time of the Withdraw member function of the ListAsLinkedList class depends on the running time of the Extract of the LinkedList<T> class. The latter was shown to be O(n) where n is the number of items in the linked list. Consequently, the total running time is O(n).


Performance Comparison: ListAsArray vs. ListAsLinkedList

The running times calculated for the various member functions of the two ordered list implementations, ListAsArray and ListAsLinkedList, are summarized below in Table gif . With the exception of two member functions, the running times of the two implementations are asymptotically identical.

Table: Running Times of Operations on Ordered Lists
 | ordered list implementation
member function | ListAsArray | ListAsLinkedList
Insert | O(1) | O(1)
IsMember | O(n) | O(n)
Find | O(n) | O(n)
Withdraw(Object&) | O(n) | O(n)
FindPosition | O(n) | O(n)
operator[](Position&) | O(1) | O(1)
operator[](unsigned int) | O(1) | O(n)
Withdraw(Position&) | O(n) | O(n)
InsertAfter | O(n) | O(1)
InsertBefore | O(n) | O(n)

The two differences are the subscripting operator which takes an integer offset, operator[] (unsigned int), and the InsertAfter function. The subscripting operation can be done constant time when using an array, but it requires O(n) in a linked list. Conversely, InsertAfter requires O(n) time when using an array, but can be done in constant time in the singly-linked list.

Table gif does not tell the whole story. The other important difference between the two implementations is the amount of space required. Consider first the array implementation, ListAsArray. An array object is comprised of two member variables--array and count. The former has type Array<Object*> and the latter is an unsigned int. The storage required for an Array<T> was computed in Chapter gif . Using that result, the storage needed for an ListAsArray which can hold at most M object pointers is given by:

Notice that since we have implemented the ordered list container using indirect storage, we have calculated the space used by the pointers to the contained objects. However, since we cannot know the types of the contained objects, we cannot calculate the space required by those objects.

A similar calculation can also be done for the ListAsLinkedList class. In this case, we assume that the actual number of contained objects is n. The total storage required is given by:

If we assume that integers and pointers require four bytes each, the storage requirement for the ListAsArray class becomes 16+4M bytes; and for the ListAsList class, 12+8n bytes. I.e., the storage needed for the array implementation is O(M), where M is the maximum length of the ordered list; whereas, the storage needed for the linked list implementation is O(n), where n is the actual number of items in the ordered list. Equating the two expressions, we get that the break-even point occurs at n=(M+1)/2. I.e., if n<(M+1)/2, the array version uses more memory space; and for n>(M+1)/2, the linked list version uses more memory space.

It is not just the amount of memory space used that should be considered when choosing an ordered list implementation. We must also consider the implications of the existence of the limit M. The array implementation requires a priori knowledge about the maximum number of items to be put in the ordered list. The total amount of storage then remains constant during the course of execution. On the other hand, the linked list version has no pre-determined maximum length. It is only constrained by the total amount of memory available to the program. Furthermore, the amount of memory used by the linked list version varies during the course of execution. We do not have to commit a large chunk of memory for the duration of the program.


Applications

The applications of lists and ordered lists are myriad. In this section we will consider only one--the use of an ordered list to represent a polynomial. In general, an [image] -order polynomial in x, for non-negative integer n, has the form

where [image]. The term [image] is the coefficient of the [image] power of x. We shall assume that the coefficients are real numbers. I.e., [image], [image].

An alternative representation for such a polynomial consists of a sequence of ordered pairs:

Each ordered pair, [image], corresponds to the term [image] of the polynomial. I.e., the ordered pair is comprised of the coefficient of the [image] term together with the subscript of that term, i. For example, the polynomial [image] can be represented by the sequence [image].

Consider now the [image] -order polynomial [image]. Clearly, there are only two nonzero coefficients: and [image]. The advantage of using the sequence of ordered pairs to represent such a polynomial is that we can omit from the sequence those pairs that have a zero coefficient. We represent the polynomial [image] by the sequence

Now that we have a way to represent polynomials, we can consider various operations on them. For example, consider the polynomial

We can compute its derivative with respect to x by differentiating each of the terms to get

where [image]. In terms of the corresponding sequences, if p(x) is represented by the sequence

then its derivative is the sequence

This result suggests a very simple algorithm to differentiate a polynomial which is represented by a sequence of ordered pairs:

  1. Drop the ordered pair that has a zero exponent.
  2. For every other ordered pair, multiply the coefficient by the exponent, and then subtract one from the exponent.
Since the representation of an [image] -order polynomial has at most n+1 ordered pairs, and since a constant amount of work is necessary for each ordered pair, this is inherently an [image] algorithm.

Of course, the worst-case running time of the polynomial differentiation will depend on the way that the sequence of ordered pairs is implemented. We will now consider an implementation that makes use of the ListAsLinkedList pointer-based implementation of ordered lists. To begin with, we need a class to represent the terms of the polynomial. Program gif gives the definition of the Term class and several of its member functions.

program10402
Program: Term Class Definition

Each Term instance has two member variables, coefficient and exponent, which correspond to the elements of the ordered pair as discussed above. The former is a double and the latter, an unsigned int.

The Term class is derived from the Object class instances of the Term class will be put into a container. Program gif gives the definitions of three member functions: a constructor, CompareTo, and Differentiate. The constructor simply takes a pair of arguments and initializes the corresponding member variables accordingly.

The CompareTo function is used to compare two Term instances. Consider two terms , [image] and [image]. We define the relation [image] on terms of a polynomial as follows:

Note that the relation [image] does not depend on the value of the variable x.

Finally, the Differentiate function does what its name says: It differentiates a term with respect to x. Given a term such as [image], it computes the result (0,0); and given a term such as [image] where i>0, it computes the result [image].

We now consider the representation of a polynomial using an ordered list. Program gif gives the definition of the class Polynomial which is derived in this case from the ListAsLinkedList class. In this example, the pointer-based implementation of lists is used.

program10438
Program: Polynomial Class Definition

Program gif defines the member function Differentiate which has the effect of changing the polynomial to its derivative with respect to x. To compute this derivative, it is necessary to call the Differentiate member function of the Term class for each term in the polynomial. Since the polynomial is implemented as a container, there is an Accept member function which can be used to perform a given operation on all of the objects in that container. In this case, we define a visitor, DifferentiatingVisitor, which assumes its argument is an instance of the Term class and differentiates it.

After the terms in the polynomial have been differentiated, it is necessary to check for the term (0,0) which arises from differentiating [image]. The Find member function is used to locate the term, and if one is found the Withdraw function is used to remove it.

The analysis of the running time of the Polynomial::Differentiate function is straightforward. The running time of Term::Differentiate is clearly O(1). So too is the running time of the function the Visit member function of the DifferentiatingVisitor. The latter function is called once for each contained object. In the worst case, given an [image] -order polynomial, there are n+1 terms. Therefore, the time required to differentiate the terms is O(n). Locating the zero term is O(n) in the worst case, and so too is deleting it. Therefore, the total running time required to differentiate a [image] -order polynomial is O(n).


Sorted Lists

The next type of searchable container that we consider is a sorted list . A sorted list is like an ordered list: It is a searchable container that holds a sequence of objects. However, the position of an item in a sorted list is not arbitrary. The items in the sequence appear in order, say, from the smallest to the largest. Of course, for such an ordering to exist, the relation used to sort the items must be a total order gif .

In addition to the basic repertoire of operations supported by all searchable containers, sorted lists provide the following operations:

FindPosition
used to find the position of an object in the sorted list;
operator []
used to access the object at a given position in the sorted list;
Withdraw
used to remove the object at a given position from the sorted list.

These operations have similar semantics to the like-named ones for ordered lists. Conspicuous by their absence are the operations InsertAfter and InsertBefore which are provided by the OrderedList class. These operations are not provided for sorted lists because they allow arbitrary insertions, but arbitrary insertions do not necessarily result in sorted lists.

Program gif gives the declaration of the class which is used to represent sorted lists: SortedList. Like its unsorted counterpart, the class SortedList is derived from the class List which is in turn derived from SearchableContainer. No new member functions are added to the inherited interface.

program10482
Program: SortedList Class Definition

Sorted lists are very similar to ordered lists. As a result, we can make use of the code for ordered lists when implementing sorted lists. Specifically, we will consider an array-based implementation of sorted lists that is derived from the ListAsArray class defined in Section gif , and a pointer-based implementation of sorted lists that is derived from the ListAsLinkedList class given in Section gif .

  • Array Implementation
  • Linked List Implementation
  • Performance Comparison: SortedListAsArray vs. SortedListAsList
  • Applications


Array Implementation

The SortedListAsArray class is declared in Program gif . Notice the use of multiple inheritance: The SortedListAsArray class is derived from both the SortedList class and the ListAsArray class. The SortedList base class is an abstract class. It provides the sorted list interface. The ListAsArray base class is a concrete class from which the sorted list class inherits much of its functionality.

program10503
Program: SortedListAsArray Class Definition

There are no addition member variables required to implement the SortedListAsArray class. I.e., the member variables provided by the base class ListAsArray are sufficient. However, there is an additional private member function declared called FindOffset. As explained below, FindOffset is used by other member functions to locate an item in the sorted list.

  • Inserting Items in a Sorted List
  • Locating Items in an Array-Binary Search
  • Finding Items in a Sorted List
  • Removing Items from a List


Adapters

An adapter converts the interface of one class into the interface expected by the user of that class. This allows a given class with an incompatible interface to be used in a situation where a different interface is expected.


Inserting Items in a Sorted List

When inserting an item into a sorted list we have as a precondition that the list is already sorted. Furthermore, once the item is inserted, we have the postcondition that the list must still be sorted. Therefore, all the items initially in the list that are larger than the item to be inserted need to be moved to the right by one position as shown in Figure gif .

figure10520
Figure: Inserting an Item into a Sorted List Implemented as an Array

Program gif defines the Insert member function for the SortedListAsArray class. This function takes as its lone argument a reference to the Object to be inserted in the list. Recall that the Insert function provided by the ListAsLinkedList class simply adds items at the end of the array. While this is both efficient and easy to implement, it is not suitable for the SortedListAsArray class since the items in the array must be end up in order.

program10737
Program: SortedListAsArray Class Insert Member Function Definition

The Insert function given in Program gif first checks that there is still room in the array for one more item. Then, to insert the item into the list, all the items in the list that are larger than the one to be inserted are moved to the right. This is accomplished by the loop on lines 5-10. Finally, a pointer to the item to be inserted is recorded in the appropriate array position on line 11.

In the worst case, the item to be inserted is smaller than all the items already in the sorted list. In this case, all [image] items must be moved one position to the right. Therefore, the running time of the Insert routine is O(n).


Locating Items in an Array-Binary Search

Given a sorted array of items, an efficient way to locate a given item is to use a binary search . The FindOffset member function of the SortedListAsArray class defined in Program gif uses a binary search to locate an item in the array which matches a given item.

program10756
Program: SortedListAsArray Class FindOffset Member Function Definition

The binary search algorithm makes use of a search interval to determine the position of an item in the sorted list. The search interval is a range of array indices in which the item being sought is expected to be found. The initial search interval is [image]. The interval is iteratively narrowed by comparing the item sought with the item found in the array at the middle of the search interval. If the middle item matches the item sought, then we are done. Otherwise, if the item sought is less than the middle item, then we can discard the middle item and the right half of the interval; if the item sought is greater than the middle item, we can discard the middle item and the left half of the interval. At each step, the size of the search interval is approximately halved. The algorithm terminates either when the item sough is found, or if the size of the search interval becomes zero.

In the worst case, the item sought is not in the sorted list. Specifically, the worst case occurs when the item sought is smaller than any item in the list because this case requires two comparisons in each iteration of the binary search loop. In the worst case, [image] iterations are required. Therefore, the running time of the FindOffset function is [image], where [image] represents the running time required to compare to Object instances. If we assume that [image], then the total running time is simply [image], where [image].


Finding Items in a Sorted List

Program gif defines the two public member functions of the SortedListAsArray class used to locate items in a sorted list. Both of these functions make use of the FindOffset member function described above.

program10774
Program: SortedListAsArray Class Find and FindPosition Member Function Definitions

The Find member function takes a const reference to an Object and returns a reference to the object contained in the sorted list which matches (i.e., compares equal to) its argument. It calls FindOffset to determine by doing a binary search the array index at which the matching object is found. Find returns a reference to the matching object, if one is found; otherwise, it returns a reference to the NullObject instance. The total running time of Find is dominated by FindOffset. Therefore, the running time is [image].

The FindPosition member function also takes a const reference an Object, but it returns a reference to a Position instead. FindPosition determines the position in the array of an object which matches its second argument.

The implementation of FindPosition is trivial: It calls FindOffset to determine the position at which the matching object is found and returns this position. The total running time of FindPosition is dominated by FindOffset. Therefore like Find, the running time of FindPosition is [image].


Removing Items from a List

The purpose of the Withdraw function is to remove an item from the sorted list. Program gif defines the Withdraw function which takes a reference to an Object.

program10810
Program: SortedListAsArray Class Withdraw Member Function Definition

The Withdraw function makes use of FindOffset to determine the array index of the item to be removed. Removing an object from position i of an ordered list which is stored in an array requires that all of the objects at positions i+1, i+2, ..., [image], be moved one position to the left. The worst case is when i=0. In this case, [image] items need to be moved to the left.

Although the Withdraw function is able to make use of FindOffset to locate the position of the item to be removed in [image] time, the total running time is dominated by the left shift, which is O(n) in the worst case. Therefore, the running time of Withdraw is O(n).


Linked List Implementation

This section presents a pointer-based implementation of sorted lists that is derived from the ListAsLinkedList class given in Section gif . The SortedListAsLinkedList class is declared in Program gif . The class definition makes use of multiple inheritance: The SortedListAsLinkedList class is derived from both the SortedList class and the ListAsLinkedList class. The former provides the sorted list interface; the latter provides almost all of the implementation.

program10834
Program: SortedListAsLinkedList Class Definition

There are no additional member variables or member functions defined in the SortedListAsLinkedList class. The inherited member variables are sufficient to implement a sorted list. In fact, the functionality inherited from the ListAsLinkedList class is almost sufficient--the only member function of which the functionality must change is the Insert operation.

  • Inserting Items in a Sorted List
  • Other Operations on Sorted Lists


Inserting Items in a Sorted List

Program gif gives the implementation of the Insert member function of the SortedListAsLinkedList class. This function takes a single argument: a reference to an Object to be inserted into the sorted list. The algorithm used for the insertion is as follows: First, the existing sorted, linked list is traversed in order to find the linked list element which is greater than or equal to the object to be inserted into the list. The traversal is done using two pointers--prevPtr and ptr. During the traversal, the latter keeps track of the current element and the former keeps track of the previous element.

By keeping track of the previous element, it is possible to efficiently insert the new item into the sorted list by calling the InsertAfter member function of the LinkedList<T> class. In Chapter gif , the InsertAfter function was shown to be O(1).

In the event that the item to be inserted is smaller than the first item in the sorted list, then rather than using the InsertAfter function, the Prepend function is used. The Prepend function was also shown to be O(1).

In the worst case, the object to be inserted into the linked list is larger than all of the objects already present in the list. In this case, the entire list needs to be traversed before doing the insertion. Consequently, the total running time for the Insert operation of the SortedListAsLinkedList class is O(n), where [image].

program10863
Program: SortedListAsLinkedList Class Insert Member Function Definition


Other Operations on Sorted Lists

Unfortunately, it is not possible to do a binary search in a linked list. As a result, it is not possible to exploit the sortedness of the list in the implementation of any of the other required operations on sorted lists. The functions inherited from the ListAsLinkedList provide all of the needed functionality.


Performance Comparison: SortedListAsArray vs. SortedListAsList

The running times calculated for the various member functions of the two sorted list implementations, SortedListAsArray and SortedListAsLinkedList, are summarized below in Table gif . With the exception of two member functions, the running times of the two implementations are asymptotically identical.

Table: Running Times of Operations on Sorted Lists
 | sorted list implementation
member function | SortedListAsArray | SortedListAsLinkedList
Insert | O(n) | O(n)
IsMember | O(n) | O(n)
Find |  | O(n)
FindPosition |  | O(n)
Withdraw | O(n) | O(n)

Neither the SortedListAsArray nor SortedListAsLinkedList implementations required any additional member variables beyond those inherited from their respective base classes, ListAsArray and ListAsLinkedList. Consequently, the space requirements analysis of the sorted list implementations is identical to that of the ordered list implementations given in Section gif .


Applications

In Section gif we saw that an [image] -order polynomial,

where [image], can be represented by a sequence of ordered pairs thus:

We also saw that it is possible to make use of an ordered list to represent such a sequence and that given such a representation, we can write an algorithm to perform differentiation.

As it turns out, the order of the terms in the sequence does not affect the differentiation algorithm. The correct result is always obtained and the running time is unaffected regardless of the order of the terms in the sequence.

Unfortunately, there are operations on polynomials whose running time depends on the order of the terms. For example, consider the addition of two polynomials:

To perform the addition all the terms involving x raised to the same power need to be grouped together.

If the terms of the polynomials are in an arbitrary order, then the grouping together of the corresponding terms is time consuming. On the other hand, if the terms are ordered, say, from smallest exponent to largest, then the summation can be done rather more efficiently. A single pass through the polynomials will suffice. It makes sense to represent each of the polynomials as a sorted list of terms using, say, the SortedListAsLinkedList class.

  • Implementation
  • Analysis


Implementation

To begin with, we need to represent the terms of the polynomial. Program gif repeats the definition of the Term class given in Section gif . The reason for repeating the definition here is that some additions are needed to support the the implementation of polynomial addition.

program10916
Program: Term Class Definition

Three additional functions are declared in Program gif . The first two, Coefficient and Exponent, are simple accessor functions which provide read-only access to the corresponding member variables of a Term instance. Clearly, the running time of each of these functions is O(1).

The third function overloads the addition operator, operator+. in such a way as to make it possible to add two Terms together. The result of the addition is another Term. The working assumption is that the terms to be added have identical exponents. If the exponents are allowed to differ, the result of of the addition is a polynomial which cannot be represented using a single term! To add terms with like exponents, we simply need to add their respective coefficients. Therefore, the running time of the Term addition operator is O(1).

We now turn to the polynomial itself. Program gif declares the Polynomial class. We have chosen in this implementation to use the pointer-based sorted list implementation to represent the sequence of terms. Therefore, the Polynomial class is derived from the SortedListAsLinkedList class.

program10947
Program: Polynomial Class Definition

Only one additional function is declared in Program gif . Specifically, the addition operator, operator+, has been overloaded to make it possible to add two Polynomials to obtain a third. Program gif gives the implementation for this operator.

program10960
Program: Polynomial Addition Operator Definition


Dedication

To my children,
Anna Kristina,
Katherine Lila
and
Alexander Edgar


Singletons

A singleton is a class of which there is only one instance. The class ensures that there only one instance is created and it provides a way to access that instance.


Analysis

The proof of the correctness of Program gif is left as an exercise for the reader (Exercise gif ). We discuss here the running time analysis of the algorithm, as there are some subtle points to remember which lead to a result that may be surprising.

Consider the addition of a polynomial p(x) with its arithmetic complement -p(x). Suppose p(x) has n terms. Clearly -p(x) also has n terms. The sum of the polynomials is the zero polynomial. An important characteristic of the zero polynomial is that it has no terms! In this case, exactly n iterations of the main loop are done (lines 7-29). Furthermore, zero iterations of the second and the third loops are required (lines 30-35 and 36-41). Since the result has no terms, there will be no calls to the Insert function. Therefore, the amount of work done in each iteration is a constant. Consequently, the best case running time is O(n).

Consider now the addition of two polynomials, p(x) and q(x), having l and m terms, respectively. Furthermore, suppose that largest exponent in p(x) is less than the smallest exponent in q(x). Consequently, there is no power of x which the two polynomials have in common. In this case, since p(x) has the lower-order terms, exactly l iterations of the main loop (lines 7-29) are done. In each of these iterations, exactly one new term is inserted into the result by calling the Insert function. Since all of the terms of p(x) will be exhausted when the main loop is finished, there will be no iterations of the second loop (lines 30-35). However, there will be exactly m iterations of the third loop (lines 36-41) in each of which one new term is inserted into the result by calling the Insert function.

Altogether, l+m calls to the Insert will be made. It was shown earlier that the running time for the insert function is O(k), where k is the number of items in the sorted list. Consequently, the total running time for the l+m insertions is

Consequently, the worst case running time for the polynomial addition given in Program gif is [image], where n=l+m. This is somewhat disappointing. The implementation is not optimal because it fails to take account of the order in which the terms of the result are computed. I.e., the Insert function repeatedly searches the sorted list for the correct position at which to insert the next term. But we know that correct position is at the end! By replacing in Program gif all of the calls to the Insert function by

result.linkedList.Append (...);
the total running time can be reduced to O(n) from [image]!


Exercises

  1. Devise an algorithm to reverse the contents of an ordered list. Determine the running time of your algorithm.
  2. Devise an algorithm to append the contents of one ordered list to the end of another. Assume that both lists are represented using arrays. What is the running time of your algorithm?
  3. Repeat Exercise gif , but this time assume that both lists are represented using linked lists. What is the running time of your algorithm?
  4. Devise an algorithm to merge the contents of two sorted lists. Assume that both lists are represented using arrays. What is the running time of your algorithm?
  5. Repeat Exercise gif , but this time assume that both lists are represented using linked lists. What is the running time of your algorithm?
  6. The Withdraw function can be used to remove items from a list one at a time. Suppose we want to provide an additional a member function, WithdrawAll, that takes one argument and withdraws all the items in a list that match the given argument.

    We can provide an abstract implementation of the WithdrawAll function in the List class like this:

    void List::WithdrawAll (Object const& arg)
{
    for (;;)
    {
        Object& object = Find (arg);
        if (object.IsNull ())
            break;
        Withdraw (object);
    }
}
    Determine the worst-case running time of this routine for each of the following cases:
    1. an array-based implementation of an ordered list,
    2. a linked-list implementation of an ordered list,
    3. an array-based implementation of a sorted list, and
    4. a linked-list implementation of a sorted list.
  7. Devise an O(n) algorithm, to remove from an ordered list all the items that match a given item. Assume the list is represented using an array.
  8. Repeat Exercise gif , but this time assume the ordered list is represented using a linked list.
  9. Consider an implementation of the OrderedList class that uses a doubly-linked list such as the one shown in Figure gif (a). Compare the running times of the operations for this implementation with those given in Table gif .
  10. Derive an expression for the amount of space used to represent an ordered list of n elements using a doubly-linked list such as the one shown in Figure gif (a). Compare this with the space used by the array-based implementation. Assume that integers and pointers each occupy four bytes.
  11. Consider an implementation of the SortedList class that uses a doubly-linked list such as the one shown in Figure gif (a). Compare the running times of the operations for this implementation with those given in Table gif .
  12. Verify that Program gif correctly computes the sum of two polynomials.
  13. Write an algorithm to multiply a polynomial by a scalar. Hint: Use a visitor.


Projects

  1. Write a visitor to solve each of the following problems:
    1. Find the smallest element of a list.
    2. Find the largest element of a list.
    3. Compute the sum of all the elements of a list.
    4. Compute the product of all the elements of a list.
  2. Design and implement a class called OrderedAsDoublyLinkedList which represents an ordered list using a doubly-linked list. Select one of the approaches shown in Figure gif .
  3. Consider the Polynomial class given in Program gif . Implement a function that computes the value of a polynomial, say p(x), for a given value of x. Hint: Use a visitor that visits all the terms in the polynomial and accumulates the result.
  4. Devise and implement an algorithm to multiply two polynomials. Hint: Consider the identity

    Write a routine to multiply a Polynomial by a Term and use the polynomial addition operator defined in Program gif .

  5. Devise and implement an algorithm to compute the [image] power of a polynomial, where k is a positive integer. What is the running time of your algorithm?
  6. For some calculations it is necessary to have very large integers, i.e., integers with an arbitrarily large number of digits. We can represent such integers using lists. Design and implement a class for representing arbitrarily large integers. Your implementation should include operations to add, subtract, and multiply such integers, and to compute the [image] power of such an integer, where k is a small positive integer. Hint: Base your design on the Polynomial class given in Program gif .


Hashing, Hash Tables and Scatter Tables

A very common paradigm in data processing involves storing information in a table and then later retrieving the information stored there. For example, consider a database of driver's license records. The database contains one record for each driver's license issued. Given a driver's license number, we can look up the information associated with that number.

Similar operations are done by the C++ compiler. The compiler uses a symbol table to keep track of the user-defined symbols in a C++ program. As it compiles a program, the compiler inserts an entry in the symbol table every time a new symbol is declared. In addition, every time a symbol is used, the compiler looks up the attributes associated with that symbol to see that it is being used correctly and to guide the generation of the executable code.

Typically the database comprises a collection of key-and-value pairs. Information is retrieved from the database by searching for a given key. In the case of the driver's license database, the key is the driver's license number and in the case of the symbol table, the key is the name of the symbol.

In general, an application may perform a large number of insertion and/or look-up operations. Occasionally it is also necessary to remove items from the database. Because a large number of operations will be done we want to do them as quickly as possible.

  • Hashing-The Basic Idea
  • Hashing Methods
  • Hash Function Implementations
  • Hash Tables
  • Scatter Tables
  • Scatter Table using Open Addressing
  • Applications
  • Exercises
  • Projects


Hashing-The Basic Idea

In this chapter we examine data structures which are designed specifically with the objective of providing efficient insertion and find operations. In order to meet the design objective, certain concessions are made. Specifically, we do not require that there be any specific ordering of the items in the container. In addition, while we still require the ability to remove items from the container, it is not our primary objective to make removal as efficient as the insertion and find operations.

Ideally we would build a data structure for which both the insertion and find operations are O(1) in the worst case. However, this kind of performance can only be achieved with complete a priori knowledge. We need to know beforehand specifically which items are to be inserted into the container. Unfortunately, we do not have this information in the general case. So, if we cannot guarantee O(1) performance in the worst case, then we make it our design objective to achieve O(1) performance in the average case.

The constant time performance objective immediately leads us to the following conclusion: Our implementation must be based in some way on an array rather than a linked list. This is because we can access the [image] element of an array in constant time, whereas the same operation in a linked list takes O(k) time.

In the previous chapter, we consider two searchable containers--the ordered list and the sorted list. In the case of an ordered list, the cost of an insertion is O(1) and the cost of the find operation is O(n). For a sorted list the cost of insertion is O(n) and the cost of the find operation is [image] for the array implementation.

Clearly, neither the ordered list nor the sorted list meets our performance objectives. The essential problem is that a search, either linear or binary, is always necessary. In the ordered list, the find operation uses a linear search to locate the item. In the sorted list, a binary search can be used to locate the item because the data is sorted. However, in order to keep the data sorted, insertion becomes O(n).

In order to meet the performance objective of constant time insert and find operations, we need a way to do them without performing a search. I.e., given an item x, we need to be able to determine directly from x the array position where it is to be stored.

  • Example
  • Keys and Hash Functions


Example

We wish to implement a searchable container which will be used to contain character strings from the set of strings K,

Suppose we define a function [image] as given by the following table:

x | h(x)
"ett" | 1
"två" | 2
"tre" | 3
"fyra" | 4
"fem" | 5
"sex" | 6
"sju" | 7
"åtta" | 8
"nio" | 9
"tio" | 10
"elva" | 11
"tolv" | 12

Then, we can implement a searchable container using an array of length n=12. To insert item x, we simply store it a position h(x)-1 of the array. Similarly, to locate item x, we simply check to see if it is found at position h(x)-1. If the function [image] can be evaluated in constant time, then the both the insert and the find operations are O(1).

We expect that any reasonable implementation of the function [image] will run in constant time, since the size of the set of strings, K, is a constant! This example illustrates how we can achieve O(1) performance in the worst case when we have complete, a priori knowledge.


Keys and Hash Functions

We are designing a container which will be used to hold some number of items of a given set K. In this context, we call the elements of the set K keys . The general approach is to store the keys in an array. The position of a key in the array is given by a function [image], called a hash function , which determines the position of a given key directly from that key.

In the general case, we expect the size of the set of keys, |K|, to be relatively large or even unbounded. E.g., if the keys are 32-bit integers, then [image]. Similarly, if the keys are arbitrary character strings of arbitrary length, then |K| is unbounded.

On the other hand, we also expect that the actual number of items stored in the container to be significantly less than |K|. I.e., if n is the number of items actually stored in the container, then [image]. Therefore, it seems prudent to use an array of size M, where M is as least as great as the maximum number of items to be stored in the container.

Consequently, what we need is a function [image]. This function maps the set of values to be stored in the container to subscripts in an array of length M. This function is called a hash function .

In general, since [image], the mapping defined by hash function will be a many-to-one mapping . I.e., there will exist many pairs of distinct keys x and y, such that [image], for which h(x)=h(y). This situation is called a collision. Several approaches for dealing with collisions are explored in the following sections.

What are the characteristics of a good hash function?

  • A good hash function avoids collisions.
  • A good hash function tends to spread keys evenly in the array.
  • A good hash function is easy to compute.

  • Avoiding Collisions
  • Spreading Keys Evenly
  • Ease of Computation


Avoiding Collisions

Ideally, given a set of [image] distinct keys, [image], the set of hash values [image] contains no duplicates. In practice, unless we know something about the keys chosen, we cannot guarantee that there will not be collisions. However, in certain applications we have some specific knowledge about the keys that we can exploit to reduce the likelihood of a collision. For example, if the keys in our application are telephone numbers, and we know that the telephone numbers are all likely to be from the same geographic area, then it makes little sense to consider the area codes in the hash function, the area codes are all likely to be the same.


Spreading Keys Evenly

Let [image] be the probability that the hash function [image]. A hash function which spreads keys evenly has the property that for [image], [image] In other words, the hash values computed by the function [image] are uniformly distributed . Unfortunately, in order to say something about the distribution of the hash values, we need to know something about the distribution of the keys.

In the absence of any information to the contrary, we assume that the keys are equiprobable. Let [image] be the set of keys that map to the value i. I.e., [image]. If this is the case, the requirement to spread the keys uniformly implies that [image]. An equal number of keys should map into each array position.


Ease of Computation

This does not mean necessarily that it is easy for someone to compute the hash function, nor does it mean that it is easy to write the algorithm to compute the function; it means that the running time of the hash function should be O(1).


The Features of C++ You Need to Know

This book does not teach the basics of programming. It is assumed that you have taken an introductory course in programming and that you have learned how to write a program in C++. I.e., you have learned the rules of C++ syntax and you have learned how to put together C++ statements in order to solve rudimentary programming problems. The following paragraphs describe more fully aspects of programming in C++ with which you should be familiar.

  • Variables
  • Parameter Passing
  • Pointers
  • Classes and Objects
  • Inheritance
  • Other Features


Hashing Methods

In this section we discuss several hashing methods. In the following discussion, we assume that we are dealing with integer-valued keys, i.e., [image]. Furthermore, we assume that the value of the hash function falls between 0 and M-1.

  • Division Method
  • Middle Square Method
  • Multiplication Method
  • Fibonacci Hashing


Division Method

Perhaps the simplest of all the methods of hashing an integer x is to divide x by M and then to use the remainder modulo M. This is called the division method of hashing . In this case, the hash function is

Generally, this approach is quite good for just about any value of M. However, in certain situations some extra care is needed in the selection of a suitable value for M. For example, it is often convenient to make M an even number. But this means that h(x) is even if x is even; and h(x) is odd if x is odd. If all possible keys are equiprobable, then this is not a problem. However if, say, even keys are more likely than odd keys, the function [image] will not spread the hashed values of those keys evenly.

Similarly, it is often tempting to let M be a power of two. E.g., [image] for some integer k>1. In this case, the hash function [image] simply extracts the bottom k bits of the binary representation of x. While this hash function is quite easy to compute, it is not a desirable function because it does not depend on all the bits in the binary representation of x.

For these reasons M is often chosen to be a prime number. For example, suppose there is a bias in the way the keys are created that makes it more likely for a key to be a multiple of some small constant, say two or three. Then making M a prime increases the likelihood that those keys are spread out evenly. Also, if M is a prime number, the division of x by that prime number depends on all the bits of x, not just the bottom k bits, for some small constant k.

The division method is extremely simple to implement. The following C++ code illustrates how to do it:

unsigned int const M = 1031; // a prime

unsigned int h (unsigned int x)
    { return x % M; }
In this case, M is a constant. However, an advantage of the division method is that M need not be a compile-time constant--its value can be determined at run time. In any event, the running time of this implementation is clearly a constant.

A potential disadvantage of the division method is due to the property that consecutive keys map to consecutive hash values:

While this ensures that consecutive keys do not collide, it does mean that consecutive array locations will be occupied. We will see that in certain implementations this can lead to degradation in performance. In the following sections we consider hashing methods that tend to scatter consecutive keys.


Middle Square Method

In this section we consider a hashing method which avoids the use of division. Since integer division is usually slower than integer multiplication, by avoiding division we can potentially improve the running time of the hashing algorithm. We can avoid division by making use of the fact that a computer does finite-precision integer arithmetic. E.g., all arithmetic is done modulo W where [image] is a power of two such that w is the word size of the computer.

The middle-square hashing method works as follows. First, we assume that M is a power of two, say [image] for some [image]. Then, to hash an integer x, we use the following hash function:

Notice that since M and W are both powers of two, the ratio [image] is also a power two. Therefore, in order to multiply the term [image] by M/W we simply shift it to the right by w-k bits! In effect, we are extracting k bits from the middle of the square of the key--hence the name of the method.

The following code fragment illustrates the middle-square method of hashing:

unsigned int const k = 10; // M==1024
unsigned int const w = bitsizeof (unsigned int);

unsigned int h (unsigned int x)
    { return (x * x) >> (w - k); }
Since x is an unsigned int, the product x * x is also an an unsigned int. If 32-bit integers are used, the product is also a 32-bit integer. The final result is obtained by shifting the product w-k bits to the right, where w is the number of bits in an integer. gif By definition, the right shift inserts zeroes on the left. Therefore, the result always falls between 0 and M-1.

The middle-square method does a pretty good job when the integer-valued keys are equiprobable. The middle-square method also has the characteristic that it scatters consecutive keys nicely. However, since the middle-square method only considers a subset of the bits in the middle of [image], keys which have a large number of leading zeroes will collide. E.g., consider the following set of keys:

This set contains all keys x such that [image]. For all of these keys h(x)=0.

A similar line of reasoning applies for keys which have a large number of trailing zeroes. Let W be an even power of two. Consider the set of keys

The least significant w/2 bits of the keys in this set are all zero. Therefore, the least significant w bits of of [image] are also zero and as a result h(x)=0 for all such keys!


Multiplication Method

A very simple variation on the middle-square method that alleviates its deficiencies is the so-called, multiplication hashing method . Instead of multiplying the key x by itself, we multiply the key by a carefully chosen constant a, and then extract the middle k bits from the result. In this case, the hashing function is

What is a suitable choice for the constant a? If we want to avoid the problems that the middle-square method encounters with keys having a large number of leading or trailing zeroes, then we should choose an a that has neither leading nor trailing zeroes.

Furthermore, if we choose an a that is relatively prime gif to W, then there exists another number a' such that [image]. In other words, a' is the inverse of a modulo W, since the product of a and its inverse is one. Such a number has the nice property that if we take a key x, and multiply it by a to get ax, we can recover the original key by multiplying the product again by a', since axa'=aa'x=1x.

There are many possible constants which the desired properties. One possibility which is suited for 32-bit arithmetic (i.e., [image]) is [image]. The binary representation of a is

This number has neither many leading nor trailing zeroes. Also, this value of a and [image] are relatively prime and the inverse of a modulo W is [image].

The following code fragment illustrates the multiplication method of hashing:

unsigned int const k = 10; // M==1024
unsigned int const w = bitsizeof (unsigned int);
unsigned int const a = 2654435769U;

unsigned int h (unsigned int x)
    { return (x * a) >> (w - k); }
Note, this implementation assumes that [image]. I.e., the natural word size of the machine is 32 bits. The code is a simple modification of the middle-square version. Nevertheless, the running time remains O(1).


Fibonacci Hashing

The final variation of hashing to be considered here is called the Fibonacci hashing method . In fact, Fibonacci hashing is exactly the multiplication hashing method discussed in the preceding section using a very special value for a. The value we choose is closely related to the number called the golden ratio.

The golden ratio is defined as follows: Given two positive numbers x and y, the ratio [image] is the golden ratio if the ratio of x to y is the same as that of x+y to x. The value of the golden ratio can be determined as follows:

There is an intimate relationship between the golden ratio and the Fibonacci numbers . In Section gif it was shown that the [image] Fibonacci number is given by

where [image] and [image]!

In the context of Fibonacci hashing, we are interested not in [image], but in the reciprocal, [image], which can be calculated as follows:

The Fibonacci hashing method is essentially the multiplication hashing method in which the constant a is chosen as the integer that is relatively prime to W which is closest to [image]. The following table gives suitable values of a for various word sizes.

W | a
 | 40503
 | 2654435769
 | 11400714819323198485

Why is [image] special? It has to do with what happens to consecutive keys when they are hashed using the multiplicative method. As shown in Figure gif , consecutive keys are spread out quite nicely. In fact, when we use [image] to hash consecutive keys, the hash value for each subsequent key falls in between the two widest spaced hash values already computed. Furthermore, it is a property of the golden ratio, [image], that each subsequent hash value divides the interval into which it falls according to the golden ratio!

figure11294
Figure: Fibonacci Hashing


Hash Function Implementations

The preceding section presents methods of hashing integer-valued keys. In reality, we cannot expect that the keys will always be integers. Depending on the application, the keys might be letters, character strings, pointers, or even more complex data structures such as Associations or Containers.

In general given a set of keys, K, and a positive constant, M, a hash function is a function of the form

In practice is it convenient to implement the hash function h as the composition of two functions f and g. The function f maps keys into integers:

where [image] is the set of non-negative integers. The function g maps non-negative integers into [image]:

Given appropriate functions f and g, the hash function h is simply defined as the composition of those functions:

I.e., the hash value of a key x is given by g(f(x)).

By decomposing the function h in this way, we can separate the problem into two parts: The first involves finding a suitable mapping from the set of keys K to the non-negative integers. The second involves mapping non-negative integers into the interval [0,M-1]. Ideally, the two problems would be unrelated. I.e., the choice of the function f would not depend on the choice of g and vice versa. Unfortunately, this is not always the case. However, if we are careful, we can design the functions in such a way that [image] is a good hash function.

The hashing methods discussed in the preceding section deal with integer-valued keys. But this is precisely the domain of the function g. Consequently, we have already examined several different alternatives for the function g. On the other hand, the choice of a suitable function for f depends on the characteristics of its domain.

In the following sections, we consider various different domains (sets of keys) and develop suitable hash functions for each of them. Each domain considered corresponds to some type or class T. For each type T, we present the definition of the function Hash declared according to the following prototype:

typedef unsigned int HashValue;

HashValue Hash (T);
Notice, we have chosen to represent the range of hash values using the type unsigned int.

  • Integral Keys
  • Floating-Point Keys
  • Character String Keys
  • Hashing Objects
  • Hashing Containers
  • Using Associations


Integral Keys

Out of all the C++ built-in data types, the so-called integral types are the simplest to hash into integers. The integral data types are char and wchart, int of all sizes, and enumerations. Since the underlying representation of such data types can be viewed as an integer, the hash function f is trivial. The only wrinkle involves dealing with negative values. Thus, a suitable function f for an integral data type is the absolute value function:

Program gif shows how to do this for plain char and int data types. Clearly, the running time of both of the functions defined in Program gif is O(1).

program11369
Program: Integral Hash Function Definitions


Floating-Point Keys

Dealing with floating-point number involves only a little more work. In C++ the floating-point data types are float, double and long double. Typically, the size of a float is 4 bytes, the size of a double is 8 bytes, and the size of a long double is 12 or 16 bytes.

We seek a function f which maps a floating-point value into a non-negative integer. One possibility is to simply reinterpret the bit pattern used to represent the floating point number as an integer. However, this is only possible when the size of the floating-point type does not exceed the size of unsigned int. This condition is typically only satisfied by the float type. gif

Another characteristic of floating-point numbers that must be dealt with is the extremely wide range of values which can be represented. E.g., when using IEEE floating-point, the smallest double precision quantity that can be represented is [image], and the largest is [image]. Somehow we need to map values in this large domain into the range of an unsigned int.

Every non-zero floating-point quantity x can be written uniquely as

where [image]. The quantity m is called the mantissa or significant and e is called the exponent . This suggests the following definition for the function f:

where [image] such that w is the word size of the machine.

This hashing method is best understood by considering the conditions under which a collision occurs between two distinct floating-point numbers x and y. Let [image] and [image] be the mantissas of x and y, respectively. The collision occurs when f(x)=f(y).

Thus, x and y collide if the magnitudes of their mantissas differ by less than 1/2W. Notice too that the exponents are not considered at all. Therefore, if x and y collide, then so too do x and [image] for all permissible values of k.

Program gif gives an implementation for the hash function defined in Equation gif . This implementation makes use of the function frexp which extracts from a double value its mantissa m and exponent e by doing efficient bit manipulations. Clearly the running time of the Hash function given in Program gif is O(1).

program11412
Program: Floating-Point Hash Function Definition


Character String Keys

Strings of characters are not a built-in C++ type. However, the standard C++ library provides the class called string which provides support for character string variables. A character string is simply a null-terminated sequence of characters. A null-terminated sequence of characters is comprised of zero or more non-null characters followed by the null character. Since such a sequence may be arbitrarily long, we have the problem again of mapping an unbounded domain into the finite range of unsigned int.

We can view a character string, s, as a sequence of n characters,

where n is the length of the string not counting the null terminator. One very simple way to hash such a string would be to simply sum the numeric values associated with each character:

As it turns out, this is not a particularly good way to hash character strings. Given that a char is universally one byte, [image] where [image], for all [image]. As a result, [image]. For example, given a string of length n=5, the value of f(s) falls between zero and 1275. In fact, the situation is even worse, the ASCII character set only uses seven bits. If the string is comprised of only ASCII characters, the result falls in the range between zero and 640.

Essentially the problem with a function f which produces a result in a relatively small interval is the situation which arises when that function is composed with the function [image]. If the size of the range of the function f is less than M, then [image] does not spread its values uniformly on the interval [0,M-1]. For example, if M=1031 only the first 640 values (62% of the range) are used!

Alternatively, suppose we have a priori knowledge that character strings are limited to length n=4. Then, we can construct an integer by concatenating the binary representations of each of the characters. I.e., given [image], we can construct an integer with the function

Since [image] is a power of two, this function is easy to write in C++:

HashValue Hash (string const& s)
    { return s[0] << 24 | s[1] << 16 | s[2] << 8 | s [3]; }
While this function certainly has a larger range, it still has two problems. First, it does not take into account the fact that the characters are usually ASCII values, and therefore do not require all eight bits. However, the more significant deficiency of this function is that it cannot deal strings of arbitrary length.

Equation gif can be generalized to deal with strings of arbitrary length as follows:

This function produces a unique integer for every possible string. Unfortunately, the range of f(s) is unbounded. A simple modification of this algorithm suffices to bound the range:

where [image] such that w is word size of the machine. Unfortunately, since W and B are both powers of two, the value computed by this hash function depends only on the last W/B characters in the character string. E.g., for [image] and [image], this result depends only on the last four characters in the string--all character strings having exactly the same last four characters collide!

We can improve matters somewhat by exploiting some of the characteristics of the ASCII code. For example, if we assume that the character string only contains 7-bit ASCII characters, then we may use [image] in Equation gif . However, this results in only a very slight improvement--for [image], the result now depends on the last four characters plus four bits of the fifth-last character.

Writing the code to compute Equation gif is actually quite straightforward if we realize that f(s) can be viewed as a polynomial in B, the coefficients of which are [image], [image], ..., [image]. Therefore, we can use Horner's rule (see Section gif ) to compute f(s) as follows:

HashValue result = 0;
for (unsigned int i = 0; s [i] != 0; ++i)
    result = result * B + s [i];
This implementation can be simplified even further if we make use of the fact that [image], where b=7. Since B is a power of two, in order to multiply the variable result by B all we need to do is to shift it left by b bits. Furthermore, having just shifted result left by b bits, we know that the least significant b bits of the result are zero. And since s[i] is a character which has no more than b=7 bits, we can replace the addition operation with an exclusive or operation.
HashValue result = 0;
for (unsigned int i = 0; s [i] != 0; ++i)
    result = result << b ^ s [i];

Of the 128 characters in the 7-bit ASCII character set, only 97 characters are printing characters including the space, tab, and newline characters (see Appendix gif ). The remaining characters are control characters which, depending on the application, rarely occur in strings. Furthermore, if we assume that letters and digits are the most common characters in strings, then only 62 of the 128 ASCII codes are used frequently. Notice, the letters (both upper and lower case) all fall between [image] and [image]. All the information is in the least significant six bits. Similarly, the digits fall between [image] and [image] --these differ in the least significant four bits. These observations suggest that using [image] should work well. I.e., for [image], the hash value depends on the last five characters plus two bits of the sixth-last character.

We have developed a hashing scheme which works quite well given strings which differ in the trailing letters. E.g., the strings "temp1", "temp2", and "temp3", all produce different hash values. However, in certain applications the strings differ in the leading letters. E.g., the two Internet domain names "ece.uwaterloo.ca" and "cs.uwaterloo.ca" collide when using Equation gif . Essentially, the effect of the characters that differ is lost because the corresponding bits have been shifted out of the hash value.

program11477
Program: Character String Hash Function Definition

This suggests a final modification which shown in Program gif . Instead of losing the b=6 most significant bits when the variable result is shifted left, we retain those bits and exclusive or them back into the shifted result variable. Using this approach, the two strings "ece.uwaterloo.ca" and "cs.uwaterloo.ca" produce different hash values.

Table gif lists a number of different character strings together with the hash values obtained using Program gif . For example, to hash the string "fyra", the following computation is performed (all numbers in octal):

1 | 4 | 6 |  |  |  |  |  |  | f
 |  | 1 | 7 | 1 |  |  |  |  | y
 |  |  |  | 1 | 6 | 2 |  |  | r
 |  |  |  |  |  | 1 | 4 | 1 | a
1 | 4 | 7 | 7 | 0 | 6 | 3 | 4 | 1 |

Table: Sample Character String Keys and the Hash Values Obtained Using Program gif
x | Hash(x) (octal)
"ett" | 01446564
"två" | 01656741
"tre" | 01656345
"fyra" | 0147706341
"fem" | 01474455
"sex" | 01624470
"sju" | 01625365
"åtta" | 0140656541
"nio" | 01575057
"tio" | 01655057
"elva" | 044556741
"tolv" | 065565566


Hashing Objects

In this section we consider hashing in the context of the object hierarchy defined in Chapter gif . The abstract base class called Object which was defined in Section gif has a const virtual member function called Hash which was declared as follows:

class Object
{
    ...
    virtual HashValue Hash () const = 0;
    ...
};
The idea is that every object instance of a class which is derived from the class Object has an associated member function called Hash which hashes that object to produce an integer HashValue. For example, given a reference to an object, Object& obj, the following computation
HashValue x = obj.Hash ();
sets the variable x to the value returned by the Hash function associated with the object to which obj refers.

A Wrapper class template was declared in Section gif which is used to wrap instances of the C++ built-in data types within an Object abstract interface. Using the Wrapper template, the four classes Char, Int, Double, and String were declared as follows:

typedef Wrapper<char> Char;
typedef Wrapper<int> Int;
typedef Wrapper<double> Double;
typedef Wrapper<string> String;
Since these classes are meant to be concrete classes, they must provide implementations for all of the member functions including the Hash function.

program11556
Program: Wrapper<T> Class Hash Member Function Definition

Program gif gives the definition of the Hash member function of the Wrapper<T> class. The implementation simply calls the appropriate hashing function from those given in Programs gif , gif and gif . For example the Wrapper<int>::Hash function calls Hash(int) and the Wrapper<string>::Hash function calls Hash(string).


Variables

You must be very comfortable with the notion of a variable as an abstraction for a region of a memory. A variable has attributes such as name, type, value, address size, lifetime and scope.


Hashing Containers

The Container class which was defined in Section gif , abstracts the notion of an object which contains other objects. The Container class is derived from the Object class and, therefore, it has a member function called Hash. What is a suitable hash function for a container?

Given a container c which contains n objects, [image], [image], ..., [image], we can define the hash function f(c) as follows:

I.e., to hash a container, simply compute the sum of the hash values of the contained objects.

Program gif shows the implementation of the Container::Hash function. This function makes use of the Accept function to cause a special visitor, HashingVisitor, to visit all of the objects contained in the container. When the HashingVisitor visits an object, it calls that object's Hash function and accumulates the result.

program11592
Program: Container Class Hash Member Function Definition

Since the Accept function is a virtual member function which is defined in the Container class. We assume that all classes derived from the Container class will provide an appropriate implementation for Accept. Note that it is not necessary for any derived class to redefine the behavior of the Hash member function--the behavior inherited from the Container class is completely generic and should suffice for all concrete container classes.

There is a slight problem with Equation gif . Different container types that happen to contain identical objects produce exactly the same hash value. E.g., an empty stack and an empty list both produce the same hash value. We have avoided this situation in Program gif by adding to the sum the value obtained from hashing the name of the container itself.


Using Associations

Hashing provides a way to determine the position of a given object directly from that object itself. Given an object x we determine its position by evaluating the appropriate hash function, h(x). We find the location of object x in exactly the same way. But of what use is this ability to find an object if, in order to compute the hash function h(x), we must be able to access the object x in the first place?

In practice, when using hashing we are dealing with keyed data . Mathematically, keyed data consists of ordered pairs

where K is a set of keys, and V is a set of values. The idea is that we will access elements of the set A using the key. I.e., the hash function for elements of the set A is given by

where [image] is the hash function associated with the set K.

For example, suppose we wish to use hashing to implement a database which contains driver's license records. Each record contains information about a driver, such as her name, address, and perhaps a summary of traffic violations. Furthermore, each record has a unique driver's license number. The driver's license number is the key and the other information is the value associated with that key.

In Section gif the class Association was declared which comprises two objects, a key and a value:

class Association : public Object, public Ownership
{
    Object* key;
    Object* value;
    ...
};
Given this declaration, the definition of the hash function for Associations is trivial. As shown in Program gif , it simply calls the Hash member function of the object to which the key member variable points.

program11623
Program: Association Class Hash Member Function Definition


Hash Tables

figure11633
Figure: Object Class Hierarchy

A hash table is a searchable container. As such, its interface provides functions for putting an object into the container, finding an object in the container, and removing an object from the container. Program gif declares the HashTable class and Program gif gives the definition of the HashTable constructor and the member function H.

program11647
Program: HashTable Class Definition

program11658
Program: HashTable Class Constructor and H Member Function Definitions

The HashTable constructor takes a single argument, and initializes the member variable length accordingly. The HashTable::H member function corresponds to the composition [image] discussed in Section gif . The member function H takes as its sole argument a const reference to an object. That object is hashed and the result of the hash modulo length is returned by H.

In the following we will consider various ways of implementing hash tables. In all cases, the underlying implementation makes use of an array. The position of an object in the array is determined by hashing the object. The main problem to be resolved is how to deal with collisions--two different objects cannot occupy the same array position at the same time. In the following section, we consider an approach which solves the problem of collisions by keeping objects that collide in a linked list.

  • Separate Chaining
  • Average Case Analysis


Separate Chaining

Figure gif shows a hash table that uses separate chaining to resolve collisions. The hash table is implemented as an array of linked lists. To insert an item into the table, it is appended to one of the linked lists. The linked list to it is appended is determined by hashing that item.

figure11681
Figure: Hash Table using Separate Chaining

Figure gif illustrates an example in which there are M=16 linked lists. The twelve character strings "ett"-"tolv" have been inserted into the table using the hashed values and in the order given in Table gif . Notice that in this example since M=16, the linked list is selected by the least significant four bits of the hashed value given in Table gif . In effect, it is only the last letter of a string which determines the linked list in which that string appears.

  • Implementation
  • Constructor and Destructor
  • Inserting and Removing Items
  • Finding an Item


Implementation

Program gif declares the class ChainedHashTable which is derived from the HashTable class. This ChainedHashTable class definition introduces one member variable called array. It is declared as an array of linked lists of pointers to Objects.

program11907
Program: ChainedHashTable Class Definition


Constructor and Destructor

The constructor and destructor for the ChainedHashTable class are defined in Program gif . The constructor takes a single argument of type unsigned int which specifies the size of hash table desired. The constructor simply initializes the HashTable base class and the array member variable accordingly. Initializing the array variable involves constructing the required number of empty linked lists. Consequently, the running time for the ChainedHashTable constructor is O(M) where M is the size of the hash table.

program11924
Program: ChainedHashTable Class Constructor, Destructor and Purge Member Function Definitions

The ChainedHashTable destructor simply calls the Purge member function. Since the ChainedHashTable is a container, the Purge function must delete any contained objects if it is the owner of those objects. Therefore, the Purge function is required to traverse each of the linked lists in the array.

To determine the running time of the Purge function, we will assume that there are n contained objects and the length of the array is M. Furthermore, let [image] be the number of items in the [image] linked list for [image]. Note that [image]. The running time of the inner loop for the [image] iteration of the outer loop is [image]. The constant overhead arises from the loop termination test which must be done at least once. The total running time of the destructor is given by

If we assume that [image], the running time simplifies to O(M+n).


Inserting and Removing Items

Program gif gives the code for inserting and removing items from a ChainedHashTable.

program11952
Program: ChainedHashTable Class Insert and Withdraw Member Function Definitions

The implementations of the Insert and Withdraw functions are remarkably simple. For example, the Insert function first calls the hash function H to compute an array index which is used to select one of the linked lists. The Append member function provided by the LinkedList<T> class is used to add a pointer to the object to the selected linked list. The total running time for the Insert operation is [image], where [image] is the running time of the Object::Hash function. Notice that if the hash function runs in constant time, then so too does hash table insertion operation!

The Withdraw function is almost identical to the Insert function. Instead of calling the Append, it calls the linked list Extract function to remove the specified object from the appropriate linked list. The running time of Withdraw is determined by the time of the Extract operation. In Chapter gif this was shown to be O(n) where n is the number of items in the linked list. In the worst case, all of the items in the ChainedHashTable have collided with each other and ended up in the same list. I.e., in the worst case if there are n items in the container, all n of them are in a single linked list. In this case, the running time of the Withdraw operation is [image].


Finding an Item

The definition of the Find member function of the ChainedHashTable class is given in Program gif . The Find function takes as its single argument a reference to an Object. The purpose of the Find operation is to return a reference to an object which is compares equal to the specified object.

program11988
Program: ChainedHashTable Class Find Member Function Definition

The Find function simply hashes its argument to select the linked list in which it should be found. Then, it traverses the linked list to locate the target object. As for the Withdraw operation, the worst case running time of the Find function occurs when all the objects in the container have collided, and the item that is being sought does not appear in the linked list. In this case, the running time of the find operation is [image].


Average Case Analysis

The previous section has shown that in the worst case, the running time to insert an object into a separately chained hash table is O(1), and the time to find or delete an object is O(n). But these bounds are no better than the same operations on plain lists! Why have we gone to all the trouble inventing hash tables?

The answer lies not in the worst-case performance, but in the average expected performance. Suppose we have a hash table of size M. Let there be exactly n items in the hash table. We call the quantity [image] the load factor . The load factor is simply the ratio of the number of items in the hash table to the array length.

Let [image] be the number of items in the [image] linked list, for [image]. The average length of a linked list is

The average length of a linked list is exactly the load factor!

If we are given the load factor [image], we can determine the average running times for the various operations. The average running time of Insert is the same as its worst case time, O(1)--this result does not depend on [image]. On the other hand, the average running time for Withdraw does depend on [image]. It is [image] since the time required to delete an item from a linked list of length [image] is [image].

To determine the average running time for the Find operation, we need to make an assumption about whether the item that is being sought is in the table. If the item is not found in the table, the search is said to be unsuccessful. The average running time for an unsuccessful search is

On the other hand, if the search target is in the table, the search is said to be successful. The average number of comparisons needed to find an arbitrary item in a linked list of length [image] is

Thus, the average running time for a successful search is

So, while any one search operation can be as bad as O(n), if we do a large number of random searches, we expect that the average running time will be [image]. In fact, if we have a sufficiently good hash function and a reasonable set of objects in the container, we can expect that those objects are distributed throughout the table. Therefore, any one search operation will not be very much worse than the worst case.

Finally, if we know how many objects will be inserted into the hash table a priori, then we can choose a table size M which is larger than the maximum number of items expected. By doing this, we can ensure that [image]. I.e., a linked list contains no more than one item on average. In this case, the average time for Withdraw is [image] and for Find it is [image].


Scatter Tables

The separately chained hash table described in the preceding section is essentially a pointer-based implementation. We have seen both pointer-based and array-based implementations for all of the data structures considered so far and hash tables are no exception. Array-based hash tables are called scatter tables .

The essential idea behind a scatter table is that all of the information is stored within a fixed size array. Hashing is used to identify the position where an item should be stored. When a collision occurs, the colliding item is stored somewhere else in the array.

One of the motivations for using scatter tables can be seen by considering again the pointer-based hash table shown in Figure gif . Since most of the linked lists are empty, much of the array is unused. At the same time, for each item that is added to the table, dynamic memory is consumed. Why not simply store the data in the unused array positions?

  • Chained Scatter Table
  • Average Case Analysis


Parameter Passing

There are two parameter passing mechanisms in C++: pass-by-value and pass-by-reference. Both of these methods are used extensively in this book. It is essential that you understand the behavioral difference between the two methods as well as the performance implications of using each of them.


Chained Scatter Table

Figure gif illustrates a chained scatter table . The elements of a chained scatter table are ordered pairs. Each array element contains a key and a pointer. All keys are stored in the table itself. Consequently, there is a fixed limit on the number of items that can be stored in a scatter table.

figure12037
Figure: Chained Scatter Table

Since the pointers point to other elements in the array, they are implemented as integer-valued array subscripts rather than as address-valued pointer variables. Since valid array subscripts start from the value zero, the null pointer must be represented not as zero, but by an integer value that is outside the array bounds.

To find an item in a chained scatter table, we begin by hashing that item to determine the location from which to begin the search. E.g., to find the string "åtta", which hashes to the value [image], we begin the search in array location [image]. The item at that location is "två", which does not match. So we follow the pointer in location [image] to location [image]. The item there, "fyra", does not match either. We follow the pointer again, this time to location [image] where we ultimately find the string we are looking for.

Comparing Figures gif and gif , we see that the chained scatter table has embedded within it the linked lists which appear to be the same as those in the separately chained hash table. However, the lists are not exactly identical. When using the chained scatter table, it is possible for lists to coalesce .

For example, when using separate chaining, the keys "tre" and "sju" appear in a separate list from the key "tolv". This is because both "tre" and "sju" hash to position [image], whereas "tolv" hashes to position [image]. The same keys appear together in a single list starting at position [image] in the chained scatter table. The two lists have coalesced.

  • Implementation
  • Constructors and Destructor
  • Inserting and Finding an Item
  • Removing Items
  • Worst-Case Running Time


Implementation

The declaration of the class ChainedScatterTable is given in Program gif . The scatter table is implemented as an array of elements of the nested type Entry. Each Entry instance has two member variables--object and next. The former is a pointer to a contained object. The latter indicates the position in the array of the next element of a chain. The value of the enumerated constant null will be used instead of zero to mark the end of a chain. The value zero is not used to mark the end of a chain because zero is a valid array subscript.

program12444
Program: ChainedScatterTable Class Definition

The ChainedScatterTable class contains a single member variable--array. It is declared as an array of ChainedScatterTable::Entry objects.


Constructors and Destructor

Program gif gives the definition of the default constructor for the ChainedScatterTable::Entry class. It shall always be the case that an unused entry will have its object pointer set to zero and its next field set to null. By default, all entries are initially unused. Therefore, the default constructor simply initializes the two member variables accordingly.

program12467
Program: ChainedScatterTable::Entry Class Constructor and ChainedScatterTable Class Constructor, Destructor and Purge Member Function Definitions

The ChainedScatterTable constructor takes a single unsigned int argument which specifies the size of scatter table desired. The constructor simply initializes the HashTable base class and the array member variable as appropriate. Initializing the array variable involves setting all of the entries to the unused state. Consequently, the running time for the ChainedScatterTable constructor is O(M) where M is the size of the scatter table.

The ChainedScatterTable destructor calls the Purge member function. The Purge function must delete any contained objects if the scatter table is the owner of those objects. The Purge function traverses the array looking for occupied entries and deletes objects as the are found. Suppose that there are n contained objects and the length of the array is M. Clearly, n cannot be greater than M since all objects are stored in the array. The running time of the Purge function is [image] and if we assume that [image], the running time simplifies to O(M).


Inserting and Finding an Item

Program gif gives the code for the Insert and Find member functions of the ChainedScatterTable class. To insert an item into a chained scatter table we need to find an unused array location in which to put the item. We first hash the item to determine the ``natural'' location for that item. If the natural location is unused, we store the item there and we are done.

program12500
Program: ChainedScatterTable Class Insert and Find Member Function Definitions

However, if the natural position for an item is occupied, then a collision has occurred and an alternate location in which to store that item must be found. When a collision occurs it must be the case that there is a chain emanating from the natural position for the item. The insertion algorithm given always adds items at the end of the chain. Therefore, after a collision has been detected, the end of the chain is found (lines 8-9).

After the end of the chain is found, an unused array position in which to store the item must be found. This is done by a simple, linear search starting from the array position immediately following the end of the chain (lines 11-13). Once an unused position is found, it is linked to the end of the chain (line 14), and the item is stored in the unused position (lines 16-17).

The worst case running time for insertion occurs when the scatter table has only one unused entry. I.e., when the number of items in the table is n=M-1, where M is the table size. In the worst case, all of the used array elements are linked into a single chain of length M-1 and the item to be inserted hashes to the head of the chain. In this case, it takes O(M) to find the end of the chain. In the worst case, the end of the chain immediately follows the unused array position. Consequently, the linear search for the unused position is also O(M). Once an unused position has been found, the actual insertion can be done in constant time. Therefore, the running time of the insertion operation is [image] in the worst case.

Program gif also gives the code for the Find member function which is used to locate a given object in the scatter table. The algorithm is straightforward. The item is hashed to find its natural location in the table. If the item is not found in the natural location but a chain emanates from that location, the chain is followed to determine if that item appears anywhere in the chain.

The worst-case running time occurs when the item for which we are looking is not in the table, the table is completely full, and all of the entries are linked together into a single linked list. In this case, the running time of the Find algorithm is [image].


Removing Items

Removing items from a chained scatter table is more complicated than putting them into the table. The goal when removing an item is to have the scatter table end up exactly as it would have appeared had that item never been inserted in the first place. Therefore, when an item is removed from the middle of a chain, items which follow it in the chain have to be moved up to fill in the hole. However, the moving-up operation is complicated by the fact that several chains may have coalesced.

Program gif gives an implementation of the Withdraw member function of the ChainedScatterTable class. The algorithm begins by checking that the table is not empty (lines 3-4). To remove an item, we first have to find it. This is what the loop on lines 5-7 does. If the item to be deleted is not in the table, when this loop terminates the variable i has the value null and an exception is thrown (lines 8-9). Otherwise, the item a position i in the table is to be removed.

program12522
Program: ChainedScatterTable Class Withdraw Member Function Definition

The purpose of the loop on lines 10-32 is to fill in the hole in the chain which results when the item at position i is removed by moving up items which follow it in the chain. What we need to do is to find the next item which follows the item at i that is safe to move into position i. The loop on lines 13-27 searches the rest of the chain following the item at i to find an item which can be safely moved.

Figure gif illustrates the basic idea. The figures shows a chained scatter table of length ten that contains integer-valued keys. There is a single chain as shown in the figure. However, notice that the values in the chain are not all equal modulo 10. In fact, this chain must have resulted from the coalescing of three chains--one which begins in position 1, one which begins in position 2, and one which begins in position 5.

figure12536
Figure: Removing Items from a Chained Scatter Table

Suppose we wish to remove item 11 which is in position 2, which is indicated by the box in Figure gif (a). To delete it, we must follow the chain to find the next item that can be moved safely up to position 2. Item 02 which follows 11 and can be moved safely up to position 2 because that is the location to which it hashes. Moving item 02 up moves the hole down the list to position 3 (Figure gif (b)). Again we follow the chain to find that item 21 can be moved safely up giving rise to the situation in Figure gif (c).

Now we have a case where an item cannot be moved. Item 05 is the next candidate to be moved. However, it is in position 5 which is the position to which it hashes. If we were to move it up, then it would no longer be in the chain which emanates from position 5. In effect, the item would no longer be accessible! Therefore, it cannot be moved safely. Instead, we must move item 31 ahead of item 5 as shown in Figure gif (d). Eventually, the hole propagates to the end of the chain, where it can be delete easily (Figure gif (e)).

The loop on lines 13-27 of Program gif finds the position j of an item which can be safely moved to position i. The algorithm makes use of the following fact: An item can be safely moved up only if it does not hash to a position which appears in the linked list between i and j. This is what the code on lines 16-24 tests.

When execution reaches line 28, either we have found an item which can be safely moved, or there does not exist such an item. If an item is found, it is moved up (lines 30-31) and we repeat the whole process again. On the other hand, if there are no more items to be moved up, then the process is finished and the main loop (lines 10-32) terminates.

The statements on lines 33-34 do the actual deed of removing the data from the position which i which by now is at the end of the chain. The final task to be done is to remove the pointer to position i, since there is no longer any data at that position. That is the job of the loop on lines 35-43.


Worst-Case Running Time

Computing a tight bound on the worst-case running time of Program gif is tricky. Assuming the item to be removed is actually in the table, then the time required to find the item (lines 3-9) is

in the worst case.

The worst-case running time of the main loop occurs when the table is full, there is only one chain, and no items can be safely moved up in the chain. In this case, the running time of the main loop (lines 10-32) is

Finally, the worst case running time of the last loop (lines 35-43) is O(M).

Therefore, the worst-case running time of the Withdraw function for chained scatter tables is

Clearly we don't want to be removing items from a chained scatter table very often!


Average Case Analysis

The previous section has shown that the worst-case running time to insert or to find an object into a chained scatter table is O(M). The average case analysis of chained scatter tables is complicated by the fact that lists coalesce. However, if we assume that chains never coalesce, then the chains which appear in a chained scatter table for a given set of items are identical to those which appear in a separately chained hash table for the same set of items.

Unfortunately we cannot assume that lists do not coalesce--they do! We therefore expect that the average list will be longer than [image] and that the running times are correspondingly slower. Knuth has shown that the average number of probes in an unsuccessful search is

and the average number of probes in a successful search is approximately

where [image] is the load factor[23]. The precise functional form of [image] and [image] is not so important here. What is important is that when [image], i.e., when the table is full, [image] and [image]. Regardless of the size of the table, an unsuccessful search requires just over two probes on average, and a successful search requires just under two probes on average!

Consequently, the average running time for insertion is

since the insertion is always done in first empty position found. Similarly, the running time for an unsuccessful search is

and for a successful search its


Scatter Table using Open Addressing

An alternative method of dealing with collisions which entirely does away with the need for links and chaining is called open addressing . The basic idea is to define a probe sequence for every key which, when followed, always leads to the key in question.

The probe sequence is essentially a sequence of functions

where [image] is a hash function, [image]. To insert item x into the scatter table, we examine array locations [image], [image], ..., until we find an empty cell. Similarly, to find item x in the scatter table we examine the same sequence of locations in the same order.

The most common probe sequences are of the form

where [image]. The function h(x) is the same hash function that we have seen before. I.e., the function h maps keys into integers in the range from zero to M-1.

The function c(i) represents the collision resolution strategy. It is required to have the following two properties:

Property 1
c(0)=0. This ensures that the first probe in the sequence is

Property 2
The set of values

must contain every integer between 0 and M-1. This second property ensures that the probe sequence eventually probes every possible array position.

  • Linear Probing
  • Quadratic Probing
  • Double Hashing
  • Implementation
  • Average Case Analysis


Linear Probing

The simplest collision resolution strategy in open addressing is called linear probing . In linear probing, the function c(i) is a linear function in i. I.e., it is of the form

Property 1 requires that c(0)=0. Therefore, [image] must be zero.

In order for [image] to satisfy Property 2, [image] and M must be relatively prime. If we know the M will always be a prime number, then any [image] will do. On the other hand, if we cannot be certain that M is prime, then [image] must be one. Therefore, linear probing sequence that is usually used is

for [image].

Figure gif illustrates an example of a scatter table using open addressing together with linear probing. For example, consider the string "åtta". This string hashes to array position [image]. The corresponding linear probing sequence begins at position [image] and goes on to positions [image], [image],.... In this case, the search for the string "åtta" succeeds after three probes.

figure13223
Figure: Scatter Table using Open Addressing and Linear Probing

To insert an item x into the scatter table, an empty cell is found by following the same probe sequence that would be used in a search for item x. Thus, linear probing finds an empty cell by doing a linear search beginning from array position h(x).

An unfortunate characteristic of linear probing arises from the fact that as the table fills, clusters of consecutive cells form and the time required for a search increases with the size of the cluster. Furthermore, when we attempt to insert an item in the table at a position which is already occupied, that item is ultimately inserted at the end of the cluster--thereby increasing its length. This by itself is not inherently a bad thing. After all, when using the chained approach, every insertion increase the length of some chain by one. However, whenever an insertion is made between two clusters that are separated by one unoccupied position, the two clusters become one, thereby potentially increasing the cluster length by an amount much greater than one--a bad thing! This phenomenon is called primary clustering .


Quadratic Probing

An alternative to linear probing that addresses the primary clustering problem is called quadratic probing . In quadratic probing, the function c(i) is a quadratic function in i. gif The general quadratic has the form

However, quadratic probing is usually done using [image].

Clearly, [image] satisfies property 1. What is not so clear is whether it satisfies property 2. In fact, in general it does not. The following theorem gives the conditions under which quadratic probing works:

Theorem When quadratic probing is used in a table of size M, where M is a prime number, the first [image] probes are distinct.

extbfProof (By contradiction). Let us assume that the theorem is false. Then there exist two distinct values i and j such that [image], that probe exactly the same position. Thus,

Since M is a prime number, the only way that the product (i-j)(i+j) can be zero modulo M is for either i-j to be zero or i+j to be zero modulo M. Since i and j are distinct, [image]. Furthermore, since both i and j are less than [image], the sum i+j is less than M. Consequently, the sum cannot be zero. We have successfully argued an absurdity--if the theorem is false one of two quantities must be zero, neither of which can possibly be zero. Therefore, the original assumption is not correct and the theorem is true.

Applying Theorem gif we get that quadratic probing works as long as the table size is prime and there are fewer than n=M/2 items in the table. In terms of the load factor [image], this occurs when [image].

Quadratic probing eliminates the primary clustering phenomenon of linear probing because instead of doing a linear search, it does a quadratic search:


Pointers

Mastering the use of pointers is essential when programming in C++. The key to understanding pointers is to recognize that a pointer variable has exactly the same set of attributes as any other C++ variable. It is crucial that you keep straight the distinctions between the value of a pointer, the address of a pointer and the object to which a pointer points.


Double Hashing

While quadratic probing does indeed eliminate the primary clustering problem, it places a restriction on the number of items that can be put in the table--the table must be less than half full. Double Hashing is yet another method of generating a probing sequence. It requires two distinct hash functions,

The probing sequence is then computed as follows

Since the collision resolution function is c(i)=ih'(x), the probe sequence depends on the key as follows: If h'(x)=1, then the probing sequence for the key x is the same as linear probing. If h'(x)=2, the probing sequence examines every other array position. This works as long as M is not even.

Clearly since c(0)=0, the double hashing method satisfies property 1. Furthermore, property 2 is satisfied as long as h'(x) and M are relatively prime. Since h'(x) can take on any value between 1 and M-1, M must be a prime number.

But what is a suitable choice for the function h'? Recall that h is defined as the composition of two functions, [image] where [image]. We can define h' as the composition [image], where

Double hashing reduces the occurrence of primary clustering since it only does a linear search if h'(x) hashes to the value 1. For a good hash function, this should only happen with probability 1/(M-1). However, for double hashing to work at all, the size of the scatter table, M, must be a prime number. Table gif summarizes the characteristics of the various open addressing probing sequences.

Table: Characteristics of the Open Addressing Probing Sequences
probing sequence | primary clustering | capacity limit | size restriction
linear probing | yes | none | none
quadratic probing | no |  | M must be prime
double hashing | no | none | M must be prime


Implementation

This section describes an implementation of a scatter table using open addressing with linear probing. Program gif declares the OpenScatterTable class. The scatter table is implemented as an array of elements of the nested type Entry. Each Entry instance has two member variables--object and state. The former is a pointer to an Object class instance. The latter is an element of the enumeration State.

program13915
Program: OpenScatterTable Class Definition

Each entry can be in one of three states--empty, occupied, or deleted. Initially, all entries are empty. When an object pointer is recorded in an entry, the state of that entry is changed to occupied. The purpose of the third state, deleted, will be discussed in conjunction with the Withdraw function below.

In addition to the lone member variable array, the OpenScatterTable class definition contains a number of private member function declarations--C, FindUnoccupied, FindMatch, and FindInstance. The member function C embodies the collision resolution strategy which is this case is linear probing. The other three member functions encapsulate functionality which is useful in implementation of the various scatter table operations.

  • Constructors and Destructor
  • Inserting Items
  • Finding Items
  • Removing Items


Constructors and Destructor

Program gif gives the definition of the default constructor for the OpenScatterTable::Entry class. It shall always be the case that an unoccupied entry will have its object pointer set to zero and the default state of all entries is initially empty. The default constructor initializes the two corresponding member variables accordingly.

program13956
Program: OpenScatterTable Class Constructor and Destructor and OpenScatterTable::Entry Class Constructor Definitions

The OpenScatterTable constructor takes a single argument of type unsigned int which specifies the size of scatter table desired. The constructor initializes the HashTable base class and the array member variable as required. Initializing the array variable requires setting all of the entries to the empty state. Consequently, the running time for the OpenScatterTable constructor is O(M) where M is the size of the scatter table.

The OpenScatterTable destructor calls the Purge member function. The Purge function must delete any contained objects if the scatter table is the owner of those objects. The Purge function traverses the array looking for occupied entries and deletes objects as the are found. If there are n contained objects and the length of the array is M, then the running time of the Purge function is [image]. Since [image], if we assume that [image] the running time becomes O(M)


Inserting Items

The procedure for inserting an item into a scatter table using open addressing is actually quite simple--find an unoccupied array location and then put the item in that location. To find an unoccupied array element, the array is probed according to a probing sequence. In this case, the probing sequence is linear probing. Program gif defines the routines needed to insert an item into the scatter table.

program13984
Program: OpenScatterTable Class C, FindUnoccupied, and Insert Member Function Definitions

The function C defines the probing sequence. As it turns out, the implementation required for a linear probing sequence is trivial. The function C is the identity function.

The purpose of the private member function FindUnoccupied is to locate an unoccupied array position. The FindUnoccupied routine probes the array according the probing sequence determined by the C function. At most n+1 probes are made, where [image] is the number of items in the scatter table. When using linear probing it is always possible to find an unoccupied cell in this many probes as long as the table is not full. Notice also that we do not search for an empty cell. Instead, the search terminates when a cell is found, the state of which is not occupied, i.e., empty or deleted. The reason for this subtlety has to do with the way items may be removed from the table. The FindUnoccupied routine returns a value between 0 and M-1, where [image] is the length of the scatter table, if an unoccupied location is found. Otherwise, it returns M to indicate that an unoccupied cell was not found.

The Insert routine takes a reference to an Object and puts that object into the scatter table. It does so by calling FindUnoccupied to determine the location of an unoccupied entry in which to put the object. The state of the unoccupied entry is set to occupied and a pointer to the object is saved in the entry.

The running time of the Insert routine is determined by that of FindUnoccupied. The worst case running time of FindUnoccupied is O(n), where n is the number of items in the scatter table. Therefore, the running time of Insert is [image].


Finding Items

The Find and FindMatch member functions of the OpenScatterTable class are defined in Program gif . The FindMatch function takes a const reference to an object and searches the scatter table for an object which matches the given one.

program14026
Program: OpenScatterTable Class FindMatch and Find Member Function Definitions

FindMatch follows the same probing sequence used by the Insert function. Therefore, if there is a matching object in the scatter table, FindMatch will make exactly the same number of probes to locate the object as were made to put the object into the table in the first place. The FindMatch routine makes at most M probes, where [image] is the size of the scatter table. However, note that the loop immediately terminates should it encounter an empty location. This is because if the target has not been found by the time an empty cell is encountered, then the target is not in the table. Notice also that the comparison is only attempted for entries which are marked occupied. Any locations marked deleted are not examined during the search but they do not terminate the search either.

The running time of the Find routine is determined by that of FindMatch. In the worst case FindMatch makes n comparisons, where n is the number of items in the table. Therefore, the running time of Find is [image].


Removing Items

Removing items from a scatter table using open addressing has to be done with some care. The naïve approach would be to locate the item to be removed and then change the state of its location to empty. However, that approach does not work! Recall that the FindMatch routine which is used to locate an item stops its search when it encounters an empty cell. Therefore, if we change the state of a cell in the middle of a cluster to empty, all subsequent searches in that cluster will stop at the empty cell. As a result, subsequent searches for an object may fail even when the target is still in the table!

One way to deal with this is to make use of the third state, deleted. Instead of marking a location empty, we mark it deleted when an item is deleted. Recall that that the FindMatch routine was written in such a way that it continues past deleted cells in its search. Also, the FindUnoccupied routine was written to stop its search when it encounters either an empty or a deleted location. Consequently, the positions marked deleted are available for reuse when insertion is done.

Program gif gives the implementation of the Withdraw. The Withdraw function takes a reference to an Object and removes it from the scatter table. It does so by first locating the specific object instance using FindInstance and then marking the location deleted. The implementation of FindInstance has been elided. It is simply a trivial variation of the FindMatch routine.

program14071
Program: OpenScatterTable Class Withdraw Member Function Definition

The running time of the Withdraw routine is determined by that of FindInstance. In the worst case FindInstance has to examine every array position. Therefore, the running time of Withdraw is [image].

There is a very serious problem with the technique of marking locations as deleted. After a large number of insertions and deletions have been done, it will be very likely that there will be no cells left that are marked empty. This is because, nowhere in any of the routines (except Purge) is a cell ever marked empty! This has the very unfortunate consequence that an unsuccessful search, i.e., a search for an object which is not in the scatter table, is [image]. Recall that FindMatch examines at most M array locations and only stops its search early when an empty location is encountered. Since there are no more empty locations, the search must examine all M locations.

If we are using the scatter table in an application in which we know a priori that no items will be removed, or perhaps only a very small number of items will be removed, then the Withdraw routine given in Program gif will suffice. However, if the application is such that a significant number of withdrawals will be made, a better implementation is required.

Ideally, when removing an item the scatter table ends up exactly as it would have appeared had that item never been inserted in the first place. Note that exactly the same constraint is met by the Withdraw function for the ChainedScatterTable class given in Program gif . It turns out that a variation of that algorithm can be used to implement the Withdraw function for the OpenScatterTable class as shown in Program gif .

program14099
Program: OpenScatterTable Class Alternate Withdraw Member Function Definition

The algorithm begins by checking that the scatter table is not empty. Then it calls FindInstance to determine the position i of the item to be removed. If the item to be removed is not in the scatter table FindInstance returns length and an exception is thrown. Otherwise, FindInstance falls between 0 and [image], which indicates that the item was found.

In the general case, the item to be deleted falls in the middle of a cluster. Deleting it would create a hole in the middle of the cluster. What we need to do is to find another item further down in the cluster which can be moved up to fill in the hole that would be created when the item at position i is deleted. The purpose of the loop on lines 11-19 is to find the position j of an item which can be moved safely into position i. Note the implementation here implicitly assumes that a linear probing sequence is used--the C function is not called explicitly. An item at position j can be moved safely to position i only if the hash value of the item at position j is not cyclically contained in the interval between i and j.

If an item is found at some position j that can be moved safely, then that item is moved to position i on line 22. The effect of moving the item at position j to position i, is to move the hole from position i to position j (line 23). Therefore, another iteration of the main loop (lines 8-24) is needed to fill in the relocated hole in the cluster.

If no item can be found to fill in the hole, then it is safe to split the cluster in two. Eventually, either because no item can be found to fill in the hole or because the hole has moved to the end of the cluster, there is nothing more to do other than to delete the hole. Thus, on lines 25-26 the entry at position i is set to empty and the associated object pointer is set to zero. Notice that the third state deleted is not required in this implementation of Withdraw.

If we use the Withdraw implementation of Program gif , the scatter table entries will only ever be in one of two two states--occupied or empty. Consequently, we can improve the bound on the worst-case for the search from [image] to [image], where n is the number of items in the scatter table.

Determining the running time of Program gif is a little tricky. Assuming the item to be deleted is actually in the table, the running time to find the position of that item (lines 3-7) is [image], where [image] is the number of item actually in the scatter table. In the worst case, the scatter table is comprised of a single cluster of n items, and we are deleting the first item of the cluster. In this case, the main loop on lines 8-24 makes one pass through the entire cluster, in the worst case moving the hole to the end of the cluster one position at at time. Thus, the running time of the main loop is [image]. The remaining lines require a constant amount of additional time. Altogether, the running time for the Withdraw function is [image] in the worst case.


Average Case Analysis

The average case analysis of open addressing is easy if we ignore the primary clustering phenomenon. Given a scatter table of size M that contains n items, we assume that each of the [image] combinations of n occupied and (m-n) empty scatter table entries is equally likely. This is the so-called uniform hashing model .

In this model we assume that the entries will either be occupied or empty, i.e., the deleted state is not used. Suppose a search for an empty cell requires exactly i probes. Then the first i-1 positions probed must have been occupied and the [image] position probed was empty. Consider the i cells which were probed. The number of combinations in which i-1 of the probed cells are occupied and one is empty is [image]. Therefore, the probability that exactly i probes are required is

The average number of probes required to find an empty cell in a table which has n occupied cells is U(n) where

Using Equation gif into Equation gif and simplifying the result gives

This result is actually quite intuitive. The load factor, [image], is the fraction of occupied entries. Therefore, [image] entries are empty so we would expect to have to probe [image] entries before finding an empty one! E.g., if the load factor is 0.75, a quarter of the entries are empty. Therefore, we expect to have to probe four entries before finding an empty one.

To calculate the average number of probes for a successful search we make the observation that when an item is initially inserted, we need to find an empty cell in which to place it. E.g., the number of probes to find the empty position into which the [image] item is to be placed is U(i). And this is exactly the number of probes it takes to find the [image] item again! Therefore, the average number of probes required for a successful search in a table which has n occupied cells is S(n) where

Substituting Equation gif in Equation gif and simplifying gives

where [image] is the [image] harmonic number (see Section gif ). Again, there is an easy intuitive derivation for this result. We can use a simple integral to calculate the mean number of probes for a successful search using the approximation [image] as follows

Empirical evidence has shown that the formulas derived for the uniform hashing model characterize the performance of scatter tables using open addressing with quadratic probing and double hashing quite well. However, they do not capture the effect of primary clustering which occurs when linear probing is used. Knuth has shown that when primary clustering is taking into account, the number of probes required to locate an empty cell is

and the number of probes required for a successful search is

The graph in Figure gif compares the predictions of the uniform hashing model (Equations gif and gif ) with the formulas derived by Knuth (Equations gif and gif ). Clearly, while the results are qualitatively similar, the formulas are in agreement for small load factors and they diverge as the load factor increases.

figure14242
Figure: Number of Probes vs. Load Factor for Uniform Hashing and Linear Probing


Applications

Hash and Scatter tables have many applications. The principal characteristic of such applications is that keyed information needs to be frequently accessed and the access pattern is either unknown or known to be random. E.g., hash tables are often used to implement the symbol table of a programming language compiler. A symbol table is used to keep track of information associated with the symbols (variable and procedure names) used by a programmer. In this case, the keys are character strings and each key hash associated with it some information about the symbol (e.g., type, address, value, lifetime, scope).

This section presents a simple application of hash and scatter tables. Suppose we are required to count the number of occurrences of each distinct word contained in a text file. We can do this easily using a hash or scatter table. Program gif gives the an implementation.

program14535
Program: Hash/Scatter Table Application--Counting Words

The class Counter is derived from the class Int defined in Section gif . In addition to all the functionality inherited from the base class, the Counter class adds the member function operator++ which increments the value by one.

The CountWords function does the actual work of counting the words in the input file. It takes as its lone argument a reference to a HashTable. Consequently, it can use any of the hash or scatter table implementations discussed in this chapter. The objects which are put into the hash table by CountWords are all instances of the class Association. Each association has as its key a String class instance, and as its value a Counter class instance.

The CountWords function reads words from the standard input file, cin, one at a time. As each word is read, a Find operation is done on the hash table to determine if there is already an association for the given key. If none is found, a new association is created an inserted into the hash table. The given word is used as the key of the new association and the value is a counter which is initialized to one. On the other hand, if there is already an association for the given word in the hash table, the corresponding counter is incremented. When it encounters the end of the input file, the CountWords function simply prints the hash table on the standard output file, cout.

The running time of the CountWords function depends on a number of factors, including the number of different keys, the frequency of occurrence of each key, and the distribution of the keys in the overall space of keys. Of course, the hash/scatter table implementation chosen has an effect as does the size of the table used. For a reasonable set of keys we expect the hash function to do a good job of spreading the keys uniformly in the table. Provided a sufficiently large table is used, the average search and insertion time is bounded by a constant. Under these ideal conditions the running time should be O(n), where n is the number of words in the input file.


Exercises

  1. Suppose we know a priori that a given key is equally likely to be any integer between a and b.
    1. When is the division method of hashing a good choice?
    2. When is the middle square method of hashing a good choice?
  2. Compute (by hand) the hash value obtained by Program gif for the strings "ece.uw.ca" and "cs.uw.ca". Hint: Refer to Appendix gif .
  3. Canadian postal codes have the format LDL DLD where L is always a letter (A-Z), D is always a digit (0-9), and is always a single space. E.g., the postal code for the University of Waterloo is N2L 3G1. Devise a suitable hash function for Canadian postal codes.
  4. For each type of hash table listed below, show the hash table obtained when we insert the keys

    in the order given into a table of size M=16 that is initially empty. Use the following table of hash values:

    x | Hash(x) (octal)
    "un" | 016456
    "deux" | 0145446470
    "trois" | 016563565063
    "quatre" | 010440656345
    "cinq" | 0142505761
    "six" | 01625070
    "sept" | 0162446164
    "huit" | 0151645064
    "neuf" | 0157446446
    "dix" | 01455070
    "onze" | 0156577345
    "douze" | 014556647345

    1. chained hash table,
    2. chained scatter table,
    3. open scatter table using linear probing,
    4. open scatter table using quadratic probing, and
    5. open scatter table using double hashing. (Use Equation gif as the secondary hash function).
  5. For each table obtained in Exercise gif , show the result when the key "deux" is withdrawn.
  6. For each table considered in Exercise gif derive an expression for the total memory space used to represent a table of size M that contains n items.
  7. Consider a chained hash table of size M that contains n items. The performance of the table decreases as the load factor [image] increases. In order to keep the load factor below one, we propose to double the size of the array when n=M. However, in order to do so we must rehash all of the elements in the table. Explain why rehashing is necessary.
  8. Give the sequence of M keys that fills a chained scatter table of size M in the shortest possible time. Find a tight, asymptotic bound on the minimum running time taken to fill the table.
  9. Give the sequence of M keys that fills a chained scatter table of size M in the longest possible time. Find a tight, asymptotic bound on the minimum running time taken to fill the table.
  10. Consider the chained hash table implementation shown in Programs gif , gif , gif and gif .
    1. Rewrite the Insert routine so that it doubles the length of the array when [image].
    2. Rewrite the Withdraw routine so that it halves the length of the array when [image].
    3. Show that the average time for both insert and withdraw operations is still O(1).
  11. Consider two sets of integers, [image] and [image].
    1. Devise an algorithm that uses a hash table to test whether S is a subset of T. What is the average running time of your algorithm?
    2. Two sets are equivalent if and only if both [image] and [image]. Show that we can test if two sets of integers are equivalent in O(m+n) time (on average).
  12. (This question should be attempted after reading Chapter gif ). Rather than use an array of linked lists, suppose we implement a hash table with an array of binary search trees.
    1. What are the worst-case running times for Insert, Find and Withdraw.
    2. What are the average running times for Insert, Find and Withdraw.
  13. (This question should be attempted after reading Section gif ). Consider a scatter table with open addressing. Devise a probe sequence of the form

    where c(i) is a full-period pseudo random number generator. Why is such a sequence likely to be better than either linear probing or quadratic probing?


Projects

  1. Complete the implementation of the ChainedHashTable class declared in Program gif by providing suitable definitions for the following member functions: IsMember, CompareTo, Accept and NewIterator. Write a test program and test your implementation.
  2. Complete the implementation of the ChainedScatterTable class declared in Program gif by providing suitable definitions for the following member functions: IsFull, IsMember, CompareTo, Accept and NewIterator. Write a test program and test your implementation.
  3. Complete the implementation of the ChainedScatterTable class declared in Program gif by providing suitable definitions for the following member functions: IsFull, IsMember, FindInstance, CompareTo, Accept and NewIterator. Write a test program and test your implementation.
  4. The Withdraw routine defined in Program gif has been written under the assumption that linear probing is used. Therefore, it does not call explicitly the collision resolution function C. Rewrite the Withdraw routine so that it works correctly regardless of the collision resolution strategy used.
  5. Consider an application that has the following profile: First, n symbols (character strings) are read in. As each symbol is read, it is assigned an ordinal number from 1 to n. Then, a large number of operations are performed. In each operation we are given either a symbol or a number and we need to determine its mate. Design, implement and test a data structure that provides both mappings in O(1) time.
  6. Spelling checkers are often implemented using hashing. However, the space required to store all the words in a complete dictionary is usually prohibitive. An alternative solution is to use a very large array of bits. The array is initialized as follows: First, all the bits are set to zero. Then for each word w in the dictionary, we set bit h(w) to one, where [image] is a suitable hash function.

    To check the spelling in a given document, we hash the words in the document one-by-one and examine the corresponding bit of the array. If the bit is a zero, the word does not appear in the dictionary and we conclude that it is misspelled. Note if the bit is a one, the word may still be misspelled, but we cannot tell.

    Design and implement a spelling checker. Hint: Use the SetAsBitVector class given in Chapter gif .


Classes and Objects

A C++ class encapsulates a set of values and a set of operations. The values are represented by the member variables of the class and the operations by the member functions of the class. In C++ a class definition introduces a new type. The instances of a class type are called objects. You should understand the special rôle of the constructor and the destructor member functions of a class and, in particular, you should know when the C++ compiler invokes each of them.


Trees

In this chapter we consider one of the most important non-linear information structures--trees. A tree is often used to represent a hierarchy . This is because the relationships between the items in the hierarchy suggest the branches of a botanical tree.

For example, a tree-like organization chart is often used to represent the lines of responsibility in a business as shown in Figure gif . The president of the company is shown at the top of the tree and the vice-presidents are indicated below her. Under the vice-presidents we find the managers and below the managers the rest of the clerks. Each clerk reports to a manager, each manager reports to a vice-president, and each vice-president reports to the president.

figure14732
Figure: Representing a Hierarchy using a Tree

It just takes a little imagination to see the tree in Figure gif . Of course, the tree is upside-down. However, this is the usual way the data structure is drawn. The president is called the root of the tree and the clerks are the leaves.

A tree is extremely useful for certain kinds of computations. For example, suppose we wish to determine the total salaries paid to employees by division or by department. The total of the salaries in division A can be found by computing the sum of the salaries paid in departments A1 and A2 plus the salary of the vice-president of division A. Similarly, the total of the salaries paid in department A1 is the sum of the salaries of the manager of department A1 and of the two clerks below her.

Clearly, in order to compute all the totals, it is necessary to consider the salary of every employee. Therefore, an implementation of this computation must visit all the employees in the tree. An algorithm that systematically visits all the items in a tree is called a tree traversal.

In this chapter we consider several different kinds of trees as well as several different tree traversal algorithms. In addition, we show how trees can be used to represent arithmetic expressions and how we can evaluate an arithmetic expression by doing a tree traversal.

  • Basics
  • N-ary Trees
  • Binary Trees
  • Tree Traversals
  • Expression Trees
  • Implementing Trees
  • Exercises
  • Projects


Basics

The following is a mathematical definition of a tree:

Definition (Tree) A tree T is a finite, non-empty set of nodes ,

with the following properties:

  1. A designated node of the set, r, is called the root of the tree; and
  2. The remaining nodes are partitioned into [image] subsets, [image], [image], ..., [image], each of which is a tree.
For convenience, we shall use the notation [image] to denote the tree T.

Notice that Definition gif is recursive--a tree is defined in terms of itself! Fortunately, we do not have a problem with infinite recursion because every tree has a finite number of of nodes and because in the base case a tree has n=0 subtrees.

It follows from Definition gif that the minimal tree is a tree comprised of a single root node. For example [image] is such a tree. When there is more than one node, the remaining nodes are partitioned into subtrees. E.g., the [image] is a tree which is comprised of the root node B and the subtree [image]. Finally, the following is also a tree

How do [image], [image] and [image] resemble their arboreal namesake? The similarity becomes apparent when we consider the graphical representation of these trees shown in Figure gif . To draw such a pictorial representation of a tree, [image], the following recursive procedure is used: First, we first draw the root node r. Then, we draw each of the subtrees, [image], [image], ..., [image], beside each other below the root. Finally, lines are drawn from r to the roots of each of the subtrees.

figure14861
Figure: Examples of Trees

Of course, trees drawn in this fashion are upside down. Nevertheless, this is the conventional way in which tree data structures are drawn. In fact, it is understood that when we speak of ``up'' and ``down,'' we do so with respect to this pictorial representation. E.g., when we move from a root to a subtree, we will say that we are moving down the tree.

The inverted pictorial representation of trees is probably due to the way that genealogical lineal charts are drawn. A lineal chart is a family tree that shows the descendants of some person. And it is from genealogy that much of the terminology associated with tree data structures is taken.

  • Terminology
  • More Terminology
  • Alternate Representations for Trees


Terminology

Consider a tree [image], [image], as given by Definition gif .

  • The degree of a node is the number of subtrees associated with that node. E.g., the degree of tree T is n.
  • A node of degree zero has no subtrees. Such a node is called a leaf .
  • Each root [image] of subtree [image] of tree T is called a child of r. The term grandchild is defined in a similar manner.
  • The root node r of tree T is the parent of all the roots [image] of the subtrees [image], [image]. The term grandparent is defined in a similar manner.
  • Two roots [image] and [image] of distinct subtrees [image] and [image] of tree T are called siblings .

Clearly the terminology used for describing tree data structures is a curious mixture of the mathematical, the genealogical, and the botanical. There is still more terminology to be introduced, but in order to do that, we need the following definition:

Definition (Path and Path Length) Given a tree T containing the set of nodes R, a path in T is defined as a non-empty sequence of nodes

where [image], for [image] such that the [image] node in the sequence, [image], is the parent of the [image] node in the sequence [image]. The length of path P is k-1.

For example, consider again the tree [image] shown in Figure gif . This tree contains many different paths. In fact, if you count carefully, you should find that there are exactly 29 distinct paths in tree [image]. This includes the path of length zero, [image]; the path of length one, [image]; and the path of length three, [image].


More Terminology

Consider a tree T containing the set of nodes R as given by Definition gif .

  • The level or depth of a node [image] in a tree T is the length of the unique path in T from its root r to the node [image]. E.g., the root of T is at level zero and the roots of the subtrees are of T are at level one.
  • The height of a node [image] in a tree T is the length of the longest path from node [image] to a leaf. Therefore, the leaves are all at height zero.
  • The height of a tree T is the height of its root node r.
  • Consider two nodes [image] and [image] in a tree T. The node [image] is an ancestor of the node [image] if there exists a path in T from [image] to [image]. Notice that [image] and [image] may be the same node. I.e., a node is its own ancestor. However, the node [image] is a proper ancestor if there exists a path p in T from [image] to [image] such that the length of the path p is non-zero.
  • Similarly, node [image] is a descendant of the node [image] if there exists a path in T from [image] to [image]. And since [image] and [image] may be the same node, a node is its own descendant. The node [image] is a proper descendant if there exists a path p in T from [image] to [image] such that the length of the path p is non-zero.


Alternate Representations for Trees

Figure gif shows an alternate representation of the tree [image] defined in Equation gif . In this case, the tree is represented as a set of nested regions in the plane. In fact, what we have is a Venn diagram which corresponds to the view that a tree is a set of sets.

figure15126
Figure: An Alternate Graphical Representation for Trees

This hierarchical, set-within-a-set view of trees is also evoked by considering the nested structure of computer programs. E.g., consider the following fragment of C++ code:

D: {
    E: {
        F: statement();
    }
    G: {
        H: {
            I: statement();
    }
        J: {
            K: statement();
            L: statement();
    }
        M: statement();
    }
}
The nesting structure of this program and the tree given in Equation gif are isomorphic . gif Therefore, it is not surprising that trees have an important rôle in the analysis and translation of computer programs.


N-ary Trees

In the preceding section we considered trees in which the nodes can have arbitrary degrees. In particular, the general case allows each of the nodes of a tree to have a different degree. In this section we consider a variation in which all of the nodes of the tree are required to have exactly the same degree.

Unfortunately, simply adding to Definition gif the additional requirement that all of the nodes of the tree have the same degree does not work. It is not possible to construct a tree which has a finite number of nodes all of which have the same degree N in any case except the trivial case of N=0. In order to make it work, we need to introduce the notion of an empty tree as follows:

Definition (N-ary Tree) An N-ary tree T is a finite set of nodes with the following properties:
  1. Either the set is empty, [image]; or
  2. The set consists of a root, R, and exactly N distinct N-ary trees. I.e., the remaining nodes are partitioned into [image] subsets, [image], [image], ..., [image], each of which is an N-ary tree such that [image].

According to Definition gif , an N-ary tree is either the empty tree, [image], or it is a non-empty set of nodes which consists of a root and exactly N subtrees. Clearly, the empty set contains neither a root, nor any subtrees. Therefore, the degree of each node of an N-ary tree is either zero or N.

There is subtle, yet extremely important consequence of Definition gif that often goes unrecognized. The empty tree, [image], is a tree. I.e., it is an object of the same type as a non-empty tree. Therefore, from the perspective of object-oriented program design, an empty tree must be an instance of some object class. It is inappropriate to use a null pointer to represent an empty tree, since a null pointer points to nothing at all!

The empty trees are called external nodes because they have no subtrees and therefore appear at the extremities of the tree. Conversely, the non-empty trees are called internal nodes .

Figure gif shows the following tertiary (N=3) trees:

In the figure, square boxes denote the empty trees and circles denote non-empty nodes. Except for the empty trees, the tertiary trees shown in the figure contain the same sets of nodes as the corresponding trees shown in Figure gif .

figure15323
Figure: Examples of N-ary Trees

Definitions gif and gif both define trees in terms of sets. In mathematics, elements of a set are normally unordered. Therefore, we might conclude that that relative ordering of the subtrees is not important. However, most practical implementations of trees define an implicit ordering of the subtrees. Consequently, it is usual to assume that the subtrees are ordered. As a result, the two tertiary trees, [image] and [image], are considered to be distinct unequal trees. Trees in which the subtrees are ordered are called ordered trees . On the other hand, trees in which the order does not matter are called oriented trees . In this book, we shall assume that all trees are ordered unless otherwise specified.

Figure gif suggests that every N-ary tree contains a significant number of external nodes. The following theorem tells us precisely how many external nodes we can expect:

Theorem An N-ary tree with [image] internal nodes contains (N-1)n+1 external nodes.

extbfProof Let the number of external nodes be l. Since every node except the root (empty or not) has a parent, there must be (n+l-1)/N parents in the tree since every parent has N children. Therefore, n=(n+l-1)/N. Rearranging this gives l=(N-1)n+1.

Since the external nodes have no subtrees, it is tempting to consider them to be the leaves of the tree. However, in the context of N-ary trees, it is customary to define a leaf node as an internal node which has only external subtrees. According to this definition, the trees shown in Figure gif have exactly the same sets of leaves as the corresponding general trees shown in Figure gif .

Furthermore, since height is defined with respect to the leaves, by having the leaves the same for both kinds of trees, the heights are also the same. The following theorem tells us something about the maximum size of a tree of a given height h:

Theorem Consider an N-ary tree T of height [image]. The maximum number of internal nodes in T is given by

extbfProof (By induction).

Base Case Consider an N-ary tree of height zero. It consists of exactly one internal node and N empty subtrees. Clearly the theorem holds for h=0 since

Inductive Hypothesis Suppose the theorem holds for [image], for some [image]. Consider a tree of height k+1. Such a tree consists of a root and N subtrees each of which contains at most [image] nodes. Therefore, altogether the number of nodes is at most

I.e., the theorem holds for k+1. Therefore, by induction on k, the theorem is true for all values of h.

An interesting consequence of Theorems gif and gif is that the maximum number of external nodes in an N-ary tree of height h is given by

The final theorem of this section addresses the maximum number of leaves in an N-ary tree of height h:

Theorem Consider an N-ary tree T of height [image]. The maximum number of leaf nodes in T is [image].

extbfProof (By induction).

Base Case Consider an N-ary tree of height zero. It consists of exactly one internal node which has N empty subtrees. Therefore, the one node is a leaf. Clearly the theorem holds for h=0 since [image].

Inductive Hypothesis Suppose the theorem holds for [image], for some [image]. Consider a tree of height k+1. Such a tree consists of a root and N subtrees each of which contains at most [image] leaf nodes. Therefore, altogether the number of leaves is at most [image]. I.e., the theorem holds for k+1. Therefore, by induction on k, the theorem is true for all values of h.


Binary Trees

In this section we consider an extremely important and useful category of tree structure--binary trees . A binary tree is an N-ary tree for which N is two. Since a binary tree is an N-ary tree, all of the results derived in the preceding section apply to binary trees. However, binary trees have some interesting characteristics that arise from the restriction that N is two. E.g., there is an interesting relationship between binary trees and the binary number system. Binary trees are also very useful for the representation of mathematical expressions involving the binary operations such as addition and multiplication.

Binary trees are defined as follows:

Definition (Binary Tree) A binary tree T is a finite set of nodes with the following properties:
  1. Either the set is empty, [image]; or
  2. The set consists of a root, r, and exactly two distinct binary trees [image] and [image], [image].
The tree [image] is called the left subtree of T, and the tree [image] is called the right subtree of T.

Binary trees are almost always considered to be ordered trees . Therefore, the two subtrees [image] and [image] are called the left and right subtrees, respectively. Consider the two binary trees shown in Figure gif . Both trees have a root with a single non-empty subtree. However, in one case it is the left subtree which is non-empty; in the other case it is the right subtree that is non-empty. Since the order of the subtrees matters, the two binary trees shown in Figure gif are different.

figure15569
Figure: Two Distinct Binary Trees

We can determine some of the characteristics of binary trees from the theorems given in the preceding section by letting N=2. E.g., Theorem gif tells us that an binary tree with [image] internal nodes contains n+1 external nodes. This result is true regardless of the shape of the tree. Consequently, we expect that the storage overhead of associated with the empty trees will be O(n).

From Theorem gif we learn that a binary tree of height [image] has at most [image] internal nodes. Conversely, the height of a binary tree with n internal nodes is at least [image]. I.e., the height of a binary tree with n nodes is [image].

Finally, according to Theorem gif , a binary tree of height [image] has at most [image] leaves. Conversely, the height of a binary tree with l leaves is at least [image]. Thus, the height of a binary tree with l leaves is


Tree Traversals

There are many different applications of trees. As a result, there are many different algorithms for manipulating them. However, many of the different tree algorithms have in common the characteristic that they systematically visit all the nodes in the tree. I.e., the algorithm walks through the tree data structure and performs some computation at each node in the tree. This process of walking through the tree is called a tree traversal .

There are essentially two different methods in which to visit systematically all the nodes of a tree--depth-first traversal and breadth-first traversal. Certain depth-first traversal methods occur frequently enough that they are given names of their own: preorder traversal, inorder traversal and postorder traversal.

The discussion that follows uses the tree in Figure gif as an example. The tree shown in the figure is a general tree in the sense of Definition gif :

However, we can also consider the tree in Figure gif to be an N-ary tree (specifically, a binary tree if we assume the existence of empty trees at the appropriate positions:

figure15734
Figure: Sample Tree

  • Preorder Traversal
  • Postorder Traversal
  • Inorder Traversal
  • Breadth-First Traversal


Preorder Traversal

The first depth-first traversal method we consider is called preorder traversal . Preorder traversal is defined recursively as follows. To do a preorder traversal of a general tree:

  1. Visit the root first; and then
  2. do a preorder traversal each of the subtrees of the root one-by-one in the order given.
Preorder traversal gets its name from the fact that it visits the root first. In the case of a binary tree, the algorithm becomes:
  1. Visit the root first; and then
  2. traverse the left subtree; and then
  3. traverse the right subtree.
For example, a preorder traversal of the tree shown in Figure gif visits the nodes in the following order:

Notice that the preorder traversal visits the nodes of the tree in precisely the same order in which they are written in Equation gif . A preorder traversal is often done when it is necessary to print a textual representation of a tree.


Postorder Traversal

The second depth-first traversal method we consider is postorder traversal . In contrast with preorder traversal, which visits the root first, postorder traversal visits the root last. To do a postorder traversal of a general tree:

  1. Do a postorder traversal each of the subtrees of the root one-by-one in the order given; and then
  2. visit the root.
To do a postorder traversal of a binary tree
  1. Traverse the left subtree; and then
  2. traverse the right subtree; and then
  3. visit the root.
A postorder traversal of the tree shown in Figure gif visits the nodes in the following order:


Inheritance

In C++ one class may be derived from another. The derived class inherits all the member variables and the member functions of the base class or classes. In addition, inherited member functions can be overridden in the derived class and new member variables and functions can be defined. You should understand how the compiler determines the code to execute when a particular member function is called.


Inorder Traversal

The third depth-first traversal method is inorder traversal . Inorder traversal only makes sense for binary trees. Whereas preorder traversal visits the root first and postorder traversal visits the root last, inorder traversal visits the root in between visiting the left and right subtrees:

  1. Traverse the left subtree; and then
  2. visit the root; and then
  3. traverse the right subtree.
An inorder traversal of the tree shown in Figure gif visits the nodes in the following order:


Breadth-First Traversal

Whereas the depth-first traversals are defined recursively, breadth-first traversal is best understood as a non-recursive traversal. The breadth-first traversal of a tree visits the nodes in the order of their depth in the tree. Breadth-first traversal first visits all the nodes at depth zero (i.e., the root), then all the nodes at depth one, and so on. At each depth the nodes are visited from left to right.

A breadth-first traversal of the tree shown in Figure gif visits the nodes in the following order:


Expression Trees

Algebraic expressions such as

have an inherent tree-like structure. For example, Figure gif is a representation of the expression in Equation gif . This kind of tree is called an expression tree .

The terminal nodes (leaves) of an expression tree are the variables or constants in the expression (a, b, c, d, and e). The non-terminal nodes of an expression tree are the operators (+, -, [image], and [image]). Notice that the parentheses which appear in Equation gif do not appear in the tree. Nevertheless, the tree representation has captured the intent of the parentheses since the subtraction is lower in the tree than the multiplication.

figure15907
Figure: Tree representing the expression a/b+(c-d)e

The common algebraic operators are either unary or binary. E.g., addition, subtraction, multiplication and division are all binary operations and negation is a unary operation. Therefore, the non-terminal nodes of the corresponding expression trees have either one or two non-empty subtrees. I.e., expression trees are usually binary trees.

What can we do with an expression tree? Perhaps the simplest thing to do is to print the expression represented by the tree. Notice that an inorder traversal of the tree in Figure gif visits the nodes in the order

Except for the missing parentheses, this is precisely the order in which the symbols appear in Equation gif !

This suggests that an inorder traversal should be used to print the expression. Consider an inorder traversal which, when it encounters a terminal node simply prints it out; and when it encounters a non-terminal node, does the following:

  1. Print a left parenthesis; and then
  2. traverse the left subtree; and then
  3. print the root; and then
  4. traverse the right subtree; and then
  5. print a right parenthesis.
Applying this procedure to the tree given in Figure gif we get

which, despite the redundant parentheses, represents exactly the same expression as Equation gif .

  • Infix Notation
  • Prefix Notation
  • Postfix Notation


Infix Notation

The algebraic expression in Equation gif is written in the usual way such mathematical expressions are written. The notation used is called infix notation because each operator appears in between its operands. As we have seen, there is a natural relationship between infix notation and inorder traversal.

Infix notation is only possible for binary operations such as addition, subtraction, multiplication, and division. Writing an operator in between its operands is possible only when it has exactly two operands. In Chapter gif we saw two alternative notations for algebraic expressions--prefix and postfix.


Prefix Notation

In prefix notation the operator is written before its operands. Therefore, in order to print the prefix expression from an expression tree, preorder traversal is done. I.e., at every non-terminal node we do the following:

  1. Print the root; and then
  2. print a left parenthesis; and then
  3. traverse the left subtree; and then
  4. print a comma; and then
  5. traverse the right subtree; and then
  6. print a right parenthesis.
If we use this procedure to print the tree given in Figure gif we get the prefix expression

While this notation may appear unfamiliar at first, consider the result obtained when we spell out the names of the operators:

plus (div (a,b), times (minus (c,d), e))
This is precisely the notation used in a typical programming language to invoke user defined procedures plus, minus, times, and div. gif


Postfix Notation

Since inorder traversal produces an infix expression and preorder traversal produces a prefix expression, it should not come as a surprise that postorder traversal produces a postfix expression. In a postfix expression, an operator always follows its operands. The beauty of postfix (and prefix) expressions is that parentheses are not necessary.

A simple postorder traversal of the tree in Figure gif gives the postfix expression

In Section gif we saw that a postfix expression is easily evaluated using a stack. So, given an expression tree, we can evaluate the expression by doing a postorder traversal to create the postfix expression and then using the algorithm given in Section gif to evaluate the expression.

In fact, it is not really necessary to first create the postfix expression before computing its value. The expression can be evaluated by making use of an evaluation stack during the course of the traversal as follows: When a terminal node is visited, its value is pushed onto the stack. When a non-terminal node is visited, two values are popped from the stack, the operation specified by the node is performed on those value, and the result is pushed back onto the evaluation stack. When the traversal terminates, there will be one result in the evaluation stack and that result is the value of the expression.

Finally, we can take this one step further. Instead of actually evaluating the expression, the code to compute the value of the expression is emitted. Again, a postorder traversal is done. However, now instead of performing the computation as each node is visited, the code needed to perform the evaluation is emitted. This is precisely what a compiler does when it compiles an expression such as Equation gif for execution.


Implementing Trees

In this section we consider the implementation of trees including general trees, N-ary trees, and binary trees. The implementations presented have been developed in the context of the abstract data type framework presented in Chapter gif . I.e., the various types of trees are viewed as classes of containers as shown in Figure gif .

figure16118
Figure: Object Class Hierarchy

Program gif declares the Tree abstract class. The Tree class encapsulates those interface elements which are common to all of the tree implementations presented in this chapter. The Tree class combines the tree interface with the container interface given in Section gif .

program16130
Program: Tree Class Definition

The Tree class adds the following functions to the public interface inherited from the Container base class:

Key
This accessor returns a reference to the object contained in the root node of a tree.
Subtree
This accessor returns a reference to the [image] subtree of the given tree.
IsEmpty
This function is a Boolean-valued accessor which returns true if the root of the tree is an empty tree, i.e., an external node.
IsLeaf
This function is a Boolean-valued accessor which returns true if the root of the tree is a leaf node.
Degree
This accessor returns the degree of the root node of the tree. The result is an unsigned int. By definition, the degree of an external node is zero.
Height
This accessor returns the height of the tree. The result is a (signed) int. By definition, the height of an empty tree is -1.
DepthFirstTraversal and BreadthFirstTraversal
These functions are analogous to the Accept member function of the container class (see Section gif ). Both of these functions perform a traversal. I.e., all the nodes of the tree are visited systematically. The former takes a reference to a PrePostVisitor and the latter takes a reference to a Visitor. When a node is visited, the appropriate functions of the visitor are applied to that node.

The preceding member functions of the Tree class are all pure virtual functions. Therefore, the Tree class is an abstract base class from which specific concrete tree classes are derived.

  • Tree Traversals
  • Tree Iterators
  • General Trees
  • N-ary Trees
  • Binary Trees
  • Binary Tree Traversals
  • Comparing Trees
  • Applications


Tree Traversals

The abstract Tree class provides default implementations for both the DepthFirstTraversal and BreadthFirstTraversal member functions. Both of these implementations call pure virtual member functions such as Key and Subtree. In effect, they are abstract algorithms . An abstract algorithm describes behavior in the absence of implementation!

  • Depth-First Traversal
  • Preorder, Inorder and Postorder Traversals
  • Breadth-First Traversal
  • Accept Member Function


Depth-First Traversal

Program gif gives the definition of the DepthFirstTraversal member function of the Tree class. The traversal routine takes one argument--a reference to an instance of the PrePostVisitor class defined in Program gif . A PrePostVisitor is a visitor with two additional member functions, PreVisit and PostVisit. During a depth-first traversal each function is called once for every node in the tree.

program16196
Program: Tree Class Traversal Member Function Definitions

The depth-first traversal routine first calls the PreVisit function with the object in the root node. Then, it calls recursively the DepthFirstTraversal function for each subtree of the given node. After all the subtrees have been visited, the PostVisit function is called. Assuming that the IsEmpty, Key and Subtree member functions all run in constant time, the total running time of the DepthFirstTraversal routine is

where n is the number of nodes in the tree, [image] is the running time of PreVisit, and [image] is the running time of [image].


Preorder, Inorder and Postorder Traversals

Preorder, inorder and postorder traversals are special cases of the more general depth-first traversal described in the preceding section. Rather than implement each of these traversals directly, we make use a design pattern pattern, called adapter , which allows the single routine to provide all the needed functionality.

Suppose we have an instance of the PuttingVisitor class (see Section gif ). As shown in Program gif , the PuttingVisitor class is derived from the abstract Visitor base class and it provides a Visit routine that prints every object it visits. However, we cannot pass a PuttingVisitor instance to the DepthFirstTraversal routine shown in Program gif because it expects a PrePostVisitor instance.

The problem is that the interface provided by the PuttingVisitor does not match the interface expected by the DepthFirstTraversal routine. The solution to this problem is to use an adapter. An adapter converts the interface provided by one class to the interface required by another. For example, if we want a preorder traversal, then the call to the PreVisit (made by DepthFirstTraversal) should be mapped to the Visit member function (provided by the PuttingVisitor). Similarly, a postorder traversal is obtained by mapping PostVisit to Visit.

program16235
Program: PrePostVisitor, PreOrder, InOrder and PostOrder Class Definitions

Program gif defines three adapter classes--PreOrder, PostOrder and InOrder. All three classes are similar: They are all derived from the Visitor abstract base class; all have a single member variable that is a reference to a Visitor class instance; and all have a constructor that takes a Visitor reference and initializes the member variable.

Each class provides a different interface mapping. For example, the PreVisit member function of the PreVisit simply calls the Visit function on the visitor member variable. Notice that the adapter provides no functionality of its own--it simply forwards member function calls to the visitor instance as required.

The following code fragment illustrates how these adapters are used:

PuttingVisitor v;
SomeTree t;
t.DepthFirstTraversal (PreOrder (v));
t.DepthFirstTraversal (InOrder (v));
t.DepthFirstTraversal (PostOrder (v));


Other Features

This book makes use of other C++ features such as templates, exceptions and run-time type information. You can learn about these topics as you work your way through the book.


Breadth-First Traversal

Program gif defines the BreadthFirstTraversal member function of the Tree class. As defined in Section gif , a breadth-first traversal of a tree visits the nodes in the order of their depth in the tree and at each level the nodes are visited from left to right.

program16281
Program: Tree Class BreadthFirstTraversal Member Function Definition

We have already seen in Section gif a non-recursive breadth-first traversal algorithm for N-ary trees. This algorithm makes use of a queue as follows. Initially, the root node of the given tree is enqueued, provided it is not the empty tree. Then, the following steps are repeated until the queue is empty:

  1. Remove the node at the head of the queue and call it head.
  2. Visit the object contained in head.
  3. Enqueue in order each non-empty subtree of head.
Notice that empty trees are never put into the queue. Furthermore, it should be obvious that each node of the tree is enqueued exactly once. Therefore, it is also dequeue exactly once. Consequently, the running time for the breadth-first traversal is [image].

There are several aspects of the way in which the queue is used here that need to be explained. First, the queue will be used to contain trees which are subtrees of the given tree. Since the subtrees of a tree are owned by the tree, they cannot also be owned by the queue. Therefore, on line 4 of Program gif , the RescindOwnership function of the queue is called. This is really conservative programming, since the queue will normally be empty by the time its destructor is called so it will should not have any contained objects to delete.

The BreadthFirstTraversal function is a const member function. Therefore, it must not modify the given tree. However, we need to push the tree onto the queue. And since the Enqueue function takes a non-const Object reference, a constcast is required on line 7 of Program gif to cast away the constness. This is not an unsafe cast in this context, because the queue is a local variable of the BreadthFirstTraversal routine and because the routine does not modify anything which it later dequeues from the queue.


Accept Member Function

The Tree class replaces the functionality provided by the single function Accept with two different kinds of traversal. Whereas the Accept function is allowed to visit the nodes of a tree in any order, the traversals visit the nodes in two different, but well-defined orders. Consequently, we have chosen to provide a default implementation of the Accept function which does a preorder traversal.

Program gif shows the implementation of the Accept member function of the Tree class. This function uses the PreOrder adapter to pass on a given visitor to the DepthFirstTraversal routine.

program16317
Program: Tree Class Accept Member Function Definition


Tree Iterators

According to the object hierarchy defined in Chapter gif , every class derived from the Container class must provide an associated Iterator class. This section describes the implementation of the Tree::Iter class which can be used to step through the contents of any tree instance.

For example, suppose we have declared a variable tree which is of type BinaryTree. Then we can view the tree instance as a container and print its contents as follows:

BinaryTree tree;
Iterator& i = tree.NewIterator;
while (!i.IsDone ()) {
    cout << *i << endl;
    ++i;
}
delete &i;
Every concrete class derived from the Container abstract base class must provide a NewIterator function. The purpose of this function is to create an instance of the appropriate type of Iterator and to associate that iterator with the corresponding container. The iterator can then be used to systematically visit the contents of the associated container.

We have already seen that when we systematically visit the nodes of a tree, we are doing a tree traversal. Therefore, the implementation of the iterator must also do a tree traversal. However, there is a catch. A recursive tree traversal routine such as DepthFirstTraversal keeps track of where it is implicitly using the processor stack. However, when we implement an iterator we must keep track of the state of the traversal explicitly. This section presents an iterator implementation which does a preorder traversal of the tree and keeps track of the current state of the traversal using a stack from Chapter gif .

Program gif gives the declaration of the Tree::Iter class. The NewIterator member function of the abstract class Tree is implemented as follows:

Iterator& Tree::NewIterator () const
    { return *new Iter (*this); }

  • Member Variables
  • Constructor and Reset Member Function
  • Operator Member Functions


Member Variables

A Tree::Iter contains two member variables--tree and stack. The former is a const reference to a Tree. Upon creation, an iterator is associated with a specific tree instance and that association remains until the iterator is destroyed. The second member variable is a reference to a Stack instance.

program16352
Program: Tree::Iter Class Definition


Constructor and Reset Member Function

The code for the Tree::Iter constructor and Reset member functions is given in Program gif . The constructor is quite simple. It takes as its lone argument a reference to a Tree instance and initializes the two member variables as follows. First, tree made to refer to the specified tree. Next a new instance of the StackAsLinkedList class is created. (The linked-list implementation of stacks is described in Section gif ). This stack will be used to contain subtrees of the given tree. Since the subtrees of a tree are owned by that tree, they cannot also be owned by the stack. Therefore the RescindOwnership function of the stack is called. Finally the Reset function is called. The running time for the constructor is O(1) assuming Reset takes a constant amount of time which indeed it does as we shall now see.

program16383
Program: Tree::Iter Class Constructor and Reset Member Function Definitions

The Reset function is called whenever it is necessary to start a new preorder traversal. It begins by calling the Purge function to make sure that the stack is empty. Then, if the associated tree is not empty, that tree is pushed onto the stack. The running time of the Reset function depends on the number of items in the stack when it is called.

Of course, the stack is initially empty. Therefore, the first time Reset is called (by the constructor) it runs in constant time. However, if the Reset function is called only after a traversal is completed, the stack will always be empty when the function is called. Therefore, under these circumstances, the running time for the Reset function is O(1).


Operator Member Functions

Program gif defines the three standard member functions for manipulating iterators, IsDone, operator*, and operator++. The purpose of IsDone is to return false as long as there are still more objects in the container which have not yet been visited. This is the case as long as there is something in the stack. Therefore, the implementation simply calls IsEmpty to test whether the stack is empty. Clearly, the running time for IsDone is O(1).

program16412
Program: Tree::Iter Class Constructor Operator Member Function Definitions

The dereferencing operator, operator*, is used to access the object which to which the iterator currently refers. In this case, the iterator refers to the object at the root of the tree which is at the top of the stack. Note, the stack will never contain an empty tree. Therefore, if the stack is not empty, then there is a referent. The dereferencing operator obtains a reference to the tree at the top of the stack by calling Top, and then calls the Key member function to access the root of the tree. The running time for operator* is O(1). When the stack is empty, this function returns a reference to the NullObject instance.

Finally, the pre-increment operator, operator++, is used to advance the iterator to the next node in a preorder traversal. It does so by popping the top tree from the stack and then pushing its subtrees onto the stack provided that they are not empty. Notice the order is important here. In a preorder traversal, the first subtree of a node is traversed before the second subtree. Therefore, the second subtree should appear in the stack below the first subtree. That is why subtrees are pushed in reverse order. The running time for operator++ is O(1).


General Trees

This section outlines an implementation of general trees in the sense of Definition gif . The salient features of the definition are first, that the nodes of a general tree have arbitrary degrees; and second, that there is no such thing as an empty tree.

The recursive nature of Definition gif has important implications when considering the implementation of such trees as containers. In effect, since a tree contains zero or more subtrees, when implemented as a container, we get a container which contains other containers! Fortunately, we have chosen by design to implement containers using indirect containment . Therefore, it is possible for a tree to contain other trees simply by keeping around pointers to those trees.

Figure gif shows the approach we have chosen for implementing general trees. This figure shows how the general tree [image] in Figure gif can be stored in memory. The basic idea is that each node has associated with it a linked list of pointers to the subtrees of that node. A linked list is used because there is no a priori restriction on its length. This allows each node to have an arbitrary degree. Furthermore, since there are no empty trees, we need not worry about representing them. An important consequence of this is that the implementation never makes use of a zero-valued tree node pointer!

figure16444
Figure: Representing General Trees using Linked Lists

Program gif declares the GeneralTree class which is used to represent general trees as specified by Definition gif . The class GeneralTree is derived from the base class Tree which is discussed in the preceding section.

program16619
Program: GeneralTree Class Definition

  • Member Variables
  • Member Functions
  • Constructor, Destructor, and Purge Member Function
  • Key and Subtree Member Functions
  • AttachSubtree and DetachSubtree Member Functions


Member Variables

The GeneralTree class definition comprises three member variables--key, degree, and list. The first, key, represents to the root node of the tree. It is a pointer to an Object. The second, an unsigned integer degree, records the degree of the root node of the tree. The third, list, is a linked list of pointers to GeneralTree instances. It is declared by instantiating the LinkedList<T> template class for [image]. This list contains pointers to the subtrees of the given tree.


Member Functions

Since the GeneralTree class is derived from the Tree base class, its interface comprises all of the public interfaces of the base classes. Furthermore, two member functions are added to the inherited interface--AttachSubtree, and DetachSubtree.

Since the GeneralTree class is a concrete class derived from the an abstract base class, it must provide implementations for all of the member functions declared as pure virtual functions in the base class. In the interest of brevity, the declarations of these functions have been elided from Program gif .


Constructor, Destructor, and Purge Member Function

Program gif defines the GeneralTree constructor, destructor, and the Purge member function. According to Definition gif , a general tree must contain at least one node--an empty tree is not allowed. Therefore, the constructor takes one argument, a reference to an Object instance. The constructor initializes the member variables as follows: The variable key is made to point at the object argument; the degree is set to zero; and, the linked list list is configured as an empty list. The running time of the constructor is clearly O(1).

program16667
Program: GeneralTree Class Constructor, Destructor and Purge Member Function Definitions

The Purge member function is inherited from the Container class interface. The purpose of Purge is to delete all contained objects and to release the associated storage if the container is the owner of those objects. Each node in a GeneralTree is a container with two varieties of contained objects--the root object and the subtrees. The approach taken in this implementation stipulates that the ownership of the root object is determined in the usual way but that all the subtrees of a given tree always are owned by that tree.

The GeneralTree class destructor simply calls the Purge member function to do its work. The Purge function first deletes the root object if it is the owner of that object. Then, it traverses the linked lists, deleting each of the attached subtrees. Finally, it deletes the linked list itself. Given a tree comprised of n nodes, the running time of the Purge function is [image] in the worst case. The factor (2n-1) is the sum of the number of contained objects deleted, n, and the number of trees deleted, n-1.


How This Book Is Organized

  • Models and Asymptotic Analysis
  • Foundational Data Structures
  • Abstract Data Types and the Class Hierarchy
  • Data Structures
  • Algorithms


Key and Subtree Member Functions

Program gif defines the various GeneralTree class member functions for manipulating general trees. The Key member function is a member variable accessor that simply returns a reference to the object contained by the root node of the tree. Clearly, its running time is O(1).

program16693
Program: GeneralTree Class Key, Subtree, AttachSubtreeand DetachSubtree Member Function Definitions

The Subtree member function takes as its lone argument an unsigned int, i, which must be between 0 and [image], where degree is the degree of the root node of the tree. It returns a reference to the [image] subtree of the given tree. The Subtree routine simply takes i steps down the linked list and returns a reference to the appropriate subtree. Assuming that i is valid, the worst case running time for Subtree is O(d), where [image] is the degree of the root node of the tree.


AttachSubtree and DetachSubtree Member Functions

Program gif also defines two functions for manipulating the subtrees of a general tree. The purpose of the AttachSubtree member function is to add the specified subtree to the root of a given tree. This function takes as its lone argument a reference to a GeneralTree instance which is to be attached. The AttachSubtree routine simply appends to the linked list a pointer to the tree to be attached and then adds one the degree variable. The running time for AttachSubtree is O(1).

Similarly, the DetachSubtree member function removes the specified subtree from the given tree. This function takes as its lone argument a reference to a GeneralTree instance which is to be removed. It removes the appropriate pointer from the linked list and then subtracts one from the degree variable. The running time for DetachSubtree is O(d) in the worst case, where [image].


N-ary Trees

We now turn to the implementation of N-ary trees as given by Definition gif . According to this definition, an N-ary tree is either an empty tree or it is a tree comprised of a root and exactly N subtrees. The implementation follows the design pattern established in the preceding section. Specifically, we view an N-ary tree as a container.

Figure gif illustrates the way in which N-ary trees can be represented. The figure gives the representation of the tertiary (N=3) tree

The basic idea is that each node has associated with it an array of length N of pointers to the subtrees of that node. An array is used because we assume that the arity of the tree, N, is known a priori.

figure16742
Figure: Representing N-ary Trees using Pointer Arrays

Notice that we explicitly represent the empty trees. I.e., a separate data structure is allocated for the representation each empty tree. Of course, an empty tree contains neither root nor subtrees.

Program gif declares the NaryTree class which represents N-ary trees as specified by Definition gif . The class NaryTree is derived from the base class Tree which, as discussed in the preceding section, combines the tree and container interfaces.

program16872
Program: NaryTree Class Definition

  • Member Variables
  • Member Functions
  • Constructors
  • IsEmpty Member Function
  • Key, AttachKey and DetachKey Member Functions
  • Subtree, AttachSubtree and DetachSubtree Member Functions


Member Variables

The implementation the NaryTree class is very similar to that of the GeneralTree class. The NaryTree class definition also comprises three member variables--key, degree, and subtree. The first, key, represents to the root node of the tree. It is a pointer to an Object. The second, an unsigned integer constant degree, records the degree of the root node of the tree. The third, subtree, is an array of pointers to NaryTree instances. It is declared by instantiating the Array<T> template class for [image]. This array contains pointers to the subtrees of the given tree.


Member Functions

Like the GeneralTree, the NaryTree class is derived from the Tree base class. Its interface comprises all of the public interfaces which are common to trees and containers. In addition the inherited functions, the following four additional member functions are defined: AttachKey, DetachKey, AttachSubtree and DetachSubtree.


Constructors

The NaryTree class declares two constructors. Implementations for the two constructors are given in Program gif . The first constructor takes a single argument of type unsigned int which specifies the degree of the tree. This constructor creates an empty tree. It does so by setting the key pointer to zero, and by setting the length of the subtree array to zero. The running time of this constructor is O(1).

program16924
Program: NaryTree Class Constructor Definitions

The second constructor takes two arguments. The first specifies the degree of the tree, and the second is a reference to an Object instance. This constructor creates a non-empty tree in which the specified object occupies the root node. According to Definition gif , every internal node in an N-ary tree must have exactly N subtrees. Therefore, this constructor creates and attaches N empty subtrees to the root node. The running time of this constructor is O(N), since N empty subtrees are created and constructed and the constructor for an empty N-ary tree takes O(1) time.


IsEmpty Member Function

The purpose of the IsEmpty accessor function is to determine whether a given N-ary tree is the empty tree. The implementation of this function is given in Program gif . In this implementation, the key pointer is zero if the tree is the empty tree. Therefore, IsEmpty returns true if when the key member variable is zero. Clearly this is a constant time, O(1), operation.

program16941
Program: NaryTree Class Member Function Definitions


Key, AttachKey and DetachKey Member Functions

Program gif also defines three functions for manipulating the root of an N-ary tree. The first, Key, is an accessor which returns a reference to the object contained in the root node of the tree. Clearly, this operation is not defined for the empty tree. If the tree is not empty, the running time of this routine is O(1).

The purpose of AttachKey is to insert the specified object into a given N-ary tree at the root node. This operation is only defined for an empty tree. The AttachKey routine takes as its lone argument a reference to the object to be inserted in the root node and makes the key member variable point at the given object. Since the node is no longer empty, it must have exactly N subtrees. Therefore, N new empty subtrees are created and attached to the node. The running time is O(N) since N subtrees are created, and the running time of the constructor for an empty N-ary tree takes O(1).

Finally, DetachKey is used to remove the object from the root of a tree. In order that the tree which remains still conforms to Definition gif , it is only permissible to remove the root from a leaf node. And upon removal, the leaf node becomes an empty tree. The implementation given in Program gif throws an exception if an attempt is made to remove the root from a non-leaf node. Otherwise, the node is a leaf which means that its N subtrees are all empty. When the root is detached, all the subtrees are deleted. The running time of this routine is clearly O(N) since there are N empty subtrees to be deleted and the cost of deleting an empty N-ary tree is constant.


Subtree, AttachSubtree and DetachSubtree Member Functions

Program gif defines the three member functions for manipulating the subtrees of an N-ary tree. The Subtree member function takes as its lone argument an unsigned int, i, which must be between 0 and N-1. It returns a reference to the [image] subtree of the given tree. Note that this operation is only defined for a non-empty N-ary tree. Given that the tree is not empty, the running time is O(1).

program16977
Program: NaryTree Class Member Function Definitions

The AttachSubtree member function takes two arguments. The first is an unsigned integer i between 0 and N-1. The second is a reference to a NaryTree instance. The purpose of this routine is to make the N-ary tree specified by the second argument become the [image] subtree of the given tree. It is only possible to attach a subtree to a non-empty node and it is only possible to attach a subtree in a place of occupied by an empty subtree. If none of the exceptions are thrown, the running time of this function is simply O(1).

The DetachSubtree member function takes a single argument i which is an unsigned integer between 0 and N-1. This routine removes the [image] subtree from a given N-ary tree and returns a reference to that subtree. Of course, it is only possible to remove a subtree from a non-empty tree. Since every non-empty node must have N subtrees, when a subtree is removed it is replaced by an empty tree. Clearly, the running time is O(1) if we assume that no exceptions are thrown.


Binary Trees

This section presents an implementation of binary trees in the sense of Definition gif . A binary tree is essentially a N-ary tree where N=2. Therefore, it is possible to implemented binary trees using the NaryTree class presented in the preceding section. However, because the NaryTree class implementation is a general implementation which can accommodate any value of N, it is somewhat less efficient in both time and space than an implementation which is designed specifically for the case N=2. Since binary trees occur quite frequently in practice, it is important to have a good implementation.

Another consequence of restricting N to two is that we can talk of the left and right subtrees of a tree. Consequently the interface provided by a binary tree class is quite different from the general interface provided by an N-ary tree class.

Figure gif shows how the binary tree given in Figure gif is be represented. The basic idea is that each node of the tree contains two pointers to the subtrees of that node. Just as we did for N-ary trees, we represent explicitly the empty trees. Since an empty tree node contains neither root nor subtrees it is represented by a structure in which all the pointers have the value zero.

figure17004
Figure: Representing Binary Trees

The BinaryTree class is declared in Program gif . It is derived from the same base class, Tree, as the classes GeneralTree and NaryTree. Therefore, it shares with those classes the common aspects of the tree and container interfaces. While the declarations of the three classes differ in the details, they all three follow a similar design pattern. Comparing Programs gif , gif and gif , we see that in addition to the constructors and destructor, they all possess similar routines for accessing and manipulating the root and the subtrees of a tree.

program17188
Program: BinaryTree Class Definition

  • Member Variables
  • Constructors
  • Destructor and Purge Member Functions


Models and Asymptotic Analysis

To analyze the performance of an algorithm, we need to have a model of the computer. Chapter gif presents a series of three models, each one less precise but easier to use than its predecessor. These models are a similar, in that they require a careful accounting of the operations performed by an algorithm.

Next, Chapter gif presents asymptotic analysis. This is an extremely useful mathematical technique because it simplifies greatly the analysis of algorithms. Asymptotic analysis obviates the need for a detailed accounting of the operations performed by an algorithm, yet at the same time gives a very general result.


Member Variables

The BinaryTree class has three member variables--key, left, and right. All three of them are pointers. The first is a pointer to an Object instance, and the latter two are pointers to BinaryTree instances. All three pointers are zero if the node represents the empty tree. Otherwise, the tree must have a root and two subtrees. Consequently, all three pointers are non-zero in a non-empty node.


Constructors

Program gif defines the two BinaryTree class constructors. The default constructor, the constructor which takes no arguments, initializes the binary tree as an empty tree. It simply sets all three member variables to zero. Clearly the running time of this constructor is O(1).

program17227
Program: BinaryTree Class Constructor Definitions

The second constructor takes as its lone argument a reference to an Object class instance. The purpose of this constructor is to create a binary tree with the specified object as its root. Since every binary tree has exactly two subtrees, the constructor creates two empty subtrees an sets the left and right member variables to point at them. The running time of this constructor is also O(1).


Destructor and Purge Member Functions

The destructor for the BinaryTree class is defined in Program gif . It simply calls the Purge member function to do the job. The Purge member function is part of the Container class interface which all container instance must provide. The purpose of the Purge routine is to delete all the owned objects in the container and to return the container to its initial empty state.

program17245
Program: BinaryTree Class Purge Member Function and Destructor Definitions

Clearly the Purge function has nothing to do if the tree is already the empty tree. If the tree is not the empty tree, then the Purge function has some cleaning-up to do. It begins by deleting the root only if the binary tree is the owner of the contained objects. Then, because a tree always owns its subtrees, the Purge routine deletes the left and right subtrees.

Suppose the binary tree contains n non-empty nodes. Theorem gif tells us that there are n+1 empty nodes. Altogether there are 2n+1=O(n) nodes. Therefore, the running time of Purge is [image] in the worst case.


Binary Tree Traversals

Program gif defines the DepthFirstTraversal member function of the BinaryTree class. This routine supports all three tree traversal methods--preorder, inorder and postorder. The implementation follows directly from the definitions given in Section gif . The traversal is implemented using recursion. I.e., the function calls itself recursively to visit the subtrees of the given node. Note that the recursion terminates properly when an empty tree is encountered since the routine does nothing in that case.

program17266
Program: BinaryTree Class DepthFirstTraversal Member Function Definition

The traversal routine takes as its lone argument a reference to a PrePostVisitor. As each node is ``visited'' during the course of the traversal, the PreVisit, Visit and PostVisit member functions of the visitor are applied to the object contained in that node.


Comparing Trees

A problem which is relatively easy to solve is determining if two trees are equivalent. Two trees are equivalent if they both have the same topology and if the objects contained in corresponding nodes are equal. Clearly, two empty trees are equivalent. Consider two non-empty binary trees [image] and [image]. Equivalence of trees is given by

A simple, recursive algorithm suffices to test the equivalence of trees.

Since the BinaryTree class is ultimately derived from the Object base class, we must provide a CompareTo member function to compare binary trees. Recall that the compare function is used to compare two objects, say obj1 and obj2 like this:

int result = obj1.CompareTo (obj2);
The CompareTo function returns a negative number if [image]; a positive number if [image]; and zero if [image].

So what we need is to define a total order relation on binary trees. Fortunately, it is possible to define such a relation for binary trees provided that the objects contained in the nodes of the trees are drawn from a totally ordered set.

Theorem Consider two binary trees [image] and [image] and the relation < given by

where [image] is either [image] or [image] and [image] is [image]. The relation < is a total order.

The proof of Theorem gif is straightforward albeit tedious. Essentially we need to show the following:

  • For any two distinct trees [image] and [image], such that [image], either [image] or [image].
  • For any three distinct trees [image], [image], and [image], if [image] and [image] then [image].
The details of the proof are left as an exercise for the reader (Exercise gif ).

Program gif gives an implementation of the CompareTo member function for the BinaryTree class. This implementation is based on the total order relation < defined in Theorem gif . The CompareTo function takes as its lone argument a const reference to an Object. However, normally that object will be another BinaryTree instance. Therefore, the dynamiccast on line 4 is normally successful.

program17339
Program: BinaryTree Class CompareTo Member Function Definition

The CompareTo function compares the two binary trees *this and arg. If they are both empty trees, CompareTo returns zero. If *this is empty and arg is not, CompareTo returns -1; and if arg is empty and *this is not, it returns 1.

Otherwise, both trees are non-empty. In this case, CompareTo first compares their respective roots. If the roots are equal, then the left subtrees are compared. Then, if the roots and the left subtrees are equal, the right subtrees are compared.

Clearly the worst-case running occurs when comparing identical trees. Suppose there are exactly n nodes in each tree. Then, the running time of the CompareTo function is [image], where [image] is the time needed to compare the objects contained in the nodes of the trees.


Applications

Section gif shows how a stack can be used to compute the value of a postfix expression such as

Suppose instead of evaluating the expression we are interested in constructing the corresponding expression tree. Once we have an expression tree, we can use the methods described in Section gif to print out the expression in prefix or infix notation. Thus, we have a means for translating expressions from one notation to another.

It turns out that an expression tree can be constructed from the postfix expression relatively easily. The algorithm to do this is a modified version of the algorithm for evaluating the expression. The symbols in the postfix expression are processed from left to right as follows:

  1. If the next symbol in the expression is an operand, a tree comprised of a single node labeled with that operand is pushed onto the stack.
  2. If the next symbol in the expression is a binary operator, the top two trees in the stack correspond to its operands. Two trees are popped from the stack and a new tree is created which has the operator as its root and the two trees corresponding to the operands as its subtrees. Then the new tree is pushed onto the stack.
After all the symbols of the expression have been processed in this fashion, the stack will contain a single tree which is the desired expression tree. Figure gif illustrates the use of a stack to construct the expression tree from the postfix expression given in Equation gif .

figure17378
Figure: Postfix to Infix Conversion using a Stack of Trees

  • Implementation


Implementation

Program gif gives the implementation of a routine, PostfixToInfix, which translates a postfix expression to an infix expression using the method described above. This routine reads an expression from the standard input file one character at at time. The expression is assumed to be a syntactically valid postfix expression comprised of single-digit numbers, single-letter variables, and the binary operators +, -, *, and /.

program17882
Program: Binary Tree Application--Postfix to Infix Conversion

Since only binary operators are allowed, the resulting expression tree is a binary tree. Consequently, the class ExpressionTree is derived from the class BinaryTree.

The main program loop, lines 13-26, reads characters from the input one at a time. If a letter or a digit is found, a new tree with the character as its root is created and pushed onto the stack (line 16). If an operator is found, a new tree is created with the operator as its root (line 19). Next, two trees are popped from the stack and attached to the new tree which is then pushed onto the stack (lines 20-24).

When the PostfixToInfix routine encounters the end-of-file, its main loop terminates. The resulting expression tree is popped from the stack, printed, and then deleted. To print the expression, the PostfixToInfix routine uses the InfixVisitor which is defined in Program gif .

program17894
Program: Binary Tree Application--Printing Infix Expressions

The InfixVisitor is intended to be used in a depth-first traversal. At each non-terminal node of the expression tree, the depth-first traversal first calls PreVisit, which prints a left parenthesis. In between the traversals of the left and right subtrees, the Visit function is called, which prints the object contained within the node. Finally, after traversing the right subtree, PostVisit prints a right parenthesis. Given the input ab/cd-e*+, the program constructs the expression tree as shown in Figure gif , and then prints the infix expression

The running time of the PostfixToInfix routine depends upon the number of symbols in the input. The running time for one iteration the main loop is O(1). Therefore, the time required to construct the expression tree given n input symbols is O(n). The DepthFirstTraversal routine visits each node of the expression tree exactly once and a constant amount of work is required to print a node. As a result, printing the infix expression is also O(n) where n is the number of input symbols.

The output expression contains all of the input symbols plus the parentheses added by the PutInfix routine. It can be shown that a valid postfix expression that contains n symbols, always has (n-1)/2 binary operators and (n+1)/2 operands (Exercise gif ). Hence, the expression tree contains (n-1)/2 non-terminal nodes and since a pair of parentheses is added for each non-terminal node in the expression tree, the output string contains 2n-1=O(n) symbols altogether. Therefore, the overall running time needed to translate a postfix expression comprised of n symbols to an infix expression is O(n).


Exercises

  1. For each tree shown in Figure gif show the order in which the nodes are visited during the following tree traversals:
    1. preorder traversal,
    2. inorder traversal (if defined),
    3. postorder traversal, and
    4. breadth-first traversal.

    figure17918
    Figure: Sample trees for Exercise gif

  2. Write a visitor that prints the nodes of a general tree in the format of Equation gif .
  3. Derive an expression for the total space needed to represent a tree of n internal nodes using each of the following classes:
    1. GeneralTree defined in Program gif ,
    2. NaryTree defined in Program gif , and
    3. BinaryTree defined in Program gif .
  4. A full node in a binary tree is a node with two non-empty subtrees. Let l be the number of leaf nodes in a binary tree. Show that the number of full nodes is l-1.
  5. The generic DepthFirstTraversal routine defined in Program gif is a recursive function. Write a non-recursive depth-first traversal routine that has exactly the same effect as the recursive version.
  6. Program gif defines a visitor that prints using infix notation the expression represented by an expression tree. Write a visitor that prints the same expression in prefix notation with the following format:

  7. Repeat Exercise gif , but this time write a visitor that the expression in postfix notation with the following format:

  8. The InfixVisitor defined in Program gif prints many redundant parentheses because it does not take into consideration the precedence of the operators. Rewrite the visitor so that it prints

    rather than

  9. Consider postfix expressions involving only binary operators. Show that if such an expression contains n symbols, it always has (n-1)/2 operators and (n+1)/2 operands.
  10. Prove Theorem gif .
  11. Generalize Theorem gif so that it applies to N-ary trees.
  12. Consider two binary trees, [image] and [image] and the relation [image] given by

    If [image], the trees are said to be isomorphic . Devise an algorithm to test whether two binary trees are isomorphic. What is the running time of your algorithm?


Projects

  1. Devise an algorithm to compute the height of a tree. Write an implementation of your algorithm as the Height member function of the abstract class Tree declared in Program gif .
  2. Devise an algorithm to count the number of internal nodes in a tree. Write an implementation of your algorithm as the Count member function of the abstract class Tree declared in Program gif .
  3. Devise an algorithm to count the number of leaves in a tree. Write an implementation of your algorithm as a member function of the abstract class Tree declared in Program gif .
  4. Devise an abstract (generic) algorithm to compare trees. (See Exercise gif ). Write an implementation of your algorithm as the CompareTo member function of the abstract class Tree declared in Program gif .
  5. The Tree::Iter class defined in Programs gif , gif and gif does a preorder traversal of a tree.
    1. Write an iterator class that does a postorder traversal.
    2. Write an iterator class that does a breadth-first traversal.
    3. Write an iterator class that does an inorder traversal. (In this case, assume that the tree is a BinaryTree).
  6. Complete the GeneralTree class declared in Program gif by providing suitable definitions for the following member functions: IsEmpty, IsLeaf, Degree and CompareTo. Write a test program and test your implementation.
  7. Complete the NaryTree class declared in Program gif by providing suitable definitions for the following member functions: NaryTree (destructor), Purge, IsLeaf, Degree and CompareTo. Write a test program and test your implementation.
  8. Complete the BinaryTree class declared in Program gif by providing suitable definitions for the following member functions: IsEmpty, IsLeaf, Degree, Key, AttachKey, DetachKey, Left, AttachLeft, DetachLeft, Right, AttachRight, DetachRight and Subtree. Write a test program and test your implementation.
  9. Write a visitor that draws a picture of a tree on the screen.
  10. Design and implement an algorithm that constructs an expression tree from an infix expression such as

    Hint: See Project gif .


Search Trees

In the preceding chapter we consider trees in which the relative positions of the nodes in the tree are unconstrained. In other words, a given item may appear anywhere in the tree. Clearly, this allows us complete flexibility in the kind of tree that we may construct. And depending on the application, this may be precisely what we need. However, if we lose track of an item, in order to find it again it may be necessary to do a complete traversal of the tree (in the worst case).

In this chapter we consider trees that are designed to support efficient search operations. In order to make it easier to search, we constrain the relative positions of the items in the tree. In addition, we show that by constraining the shape of the tree as well as the relative positions of the items in the tree, search operations can be made even more efficient.

  • Basics
  • Searching a Search Tree
  • Average Case Analysis
  • Implementing Search Trees
  • AVL Search Trees
  • M-Way Search Trees
  • B-Trees
  • Applications
  • Exercises
  • Projects


Preface

This book was motivated by my experience in teaching the course E&CE 250: Algorithms and Data Structures in the Computer Engineering program at the University of Waterloo. I have observed that the advent of object-oriented methods and the emergence of object-oriented design patterns has lead to a profound change in the pedagogy of data structures and algorithms. The successful application of these techniques gives rise to a kind of cognitive unification: Ideas that are disparate and apparently unrelated seem to come together when the appropriate design patterns and abstractions are used.

This paradigm shift is both evolutionary and revolutionary. On the one hand, the knowledge base grows incrementally as programmers and researchers invent new algorithms and data structures. On the other hand, the proper use of object-oriented techniques requires a fundamental change in the way the programs are designed and implemented. Programmers who are well schooled in the procedural ways often find the leap to objects to be a difficult one.

  • Goals
  • Approach
  • Outline
  • Suggested Course Outline
  • Online Course Materials


Foundational Data Structures

When implementing a data structure, we must decide first whether to use an array or a linked list as the underlying organizational technique. For this reason, the array and the linked list are called foundational data structures. Chapter gif also covers multi-dimensional arrays and matrices.


Basics

A tree which supports efficient search, insertion, and withdrawal operations is called a search tree . In this context the tree is used to store a finite set of keys drawn from a totally ordered set of keys K. Each node of the tree contains one or more keys and all the keys in the tree are unique, i.e., no duplicate keys are permitted.

What makes a tree into a search tree is that the keys do not appear in arbitrary nodes of the tree. Instead, there is a data ordering criterion which determines where a given key may appear in the tree in relation to the other keys in that tree. The following sections present two related types of search trees, M-way search trees and binary search trees.

  • M-Way Search Trees
  • Binary Search Trees


M-Way Search Trees

Definition (M-way Search Tree) An M-way search tree T is a finite set of keys. Either the set is empty, [image]; or the set consists of n M-way subtrees [image], [image], ..., [image], and n-1 keys, [image], [image], ..., [image],

where [image], such that the keys and nodes satisfy the following data ordering properties :

  1. The keys in each node are distinct and ordered, i.e., [image] for [image].
  2. All the keys contained in subtree [image] are less than [image], i.e., [image] for [image]. The tree [image] is called the left subtree with respect to the key [image].
  3. All the keys contained in subtree [image] are greater than [image], i.e., [image] for [image]. The tree [image] is called the right subtree with respect to the key [image].

Figure gif gives an example of an M-way search tree for M=4. In this case, each of the non-empty nodes of the tree has between one and three keys and at most four subtrees. All the keys in the tree satisfy the data ordering properties. Specifically, the keys in each node are ordered and for each key in the tree, all the keys in the left subtree with respect to the given key are are less than the given key, and all the keys in the right subtree with respect to the given key are larger than than the given key. Finally, it is important to note that the topology of the tree is not determined by the particular set of keys it contains.

figure18704
Figure: An M-way Search Tree


Binary Search Trees

Just as the binary tree is an important category of N-ary trees, the binary search tree is an important category of M-way search trees.

Definition (Binary Search Tree) A binary search tree T is a finite set of keys. Either the set is empty, [image]; or the set consists of a root r and exactly two binary search trees [image] and [image], [image], such that the following properties are satisfied:
  1. All the keys contained in left subtree, [image], are less than r, i.e., [image].
  2. All the keys contained in the right subtree, [image], are greater than r, i.e., [image].

Figure gif shows an example of a binary search tree. In this case, since the nodes of the tree carry alphabetic rather than numeric keys, the ordering of the keys is alphabetic. I.e., all the keys in the left subtree of a given node precede alphabetically the root of the that node, and all the keys in the right subtree of a given node follow alphabetically the root of that node. The empty trees are shown explicitly as boxes in Figure gif . However, in order to simplify the graphical representation, the empty trees are often omitted from the diagrams.

figure18764
Figure: A Binary Search Tree


Searching a Search Tree

The main advantage of a search tree is that the data ordering criterion ensures that it is not necessary to do a complete tree traversal in order to locate a given item. Since search trees are defined recursively, it is easy to define a recursive search method.

  • Searching an M-way Tree
  • Searching a Binary Tree


Searching an M-way Tree

Consider the search for a particular item, say x, in an M-way search tree. The search always begins at the root. If the tree is empty, the search fails. Otherwise, the keys contained in the root node are examined to determine if the object of the search is present. If it is, the search terminates successfully. If it is not, there are three possibilities: Either the object of the search, x, is less than [image], in which case subtree [image] is searched; or x is greater than [image], in which case subtree [image] is searched; or there exists an i such that [image] for which [image], in which case subtree [image] is searched.

Notice that when x is not found in a given node, only one of the n subtrees of that node is searched. Therefore, a complete tree traversal is not required. A successful search begins at the root and traces a downward path in the tree, which terminates at the node containing the object of the search. Clearly, the running time of a successful search is determined by the depth in the tree of object of the search.

When the object of the search is not in the search tree, the search method described above traces a downward path from the root which terminates when an empty subtree is encountered. In the worst case, the search path passes through the deepest leaf node. Therefore, the worst-case running time for an unsuccessful search is determined by the height of the search tree.


Searching a Binary Tree

The search method described above applies directly to binary search trees. As above, the search begins at the root node of the tree. If the object of the search, x, matches the root r, the search terminates successfully. If it does not, then if x is less than r, the left subtree is searched; otherwise x must be greater than r, in which case the right subtree is searched.

Figure gif shows two binary search trees. The tree [image] is an example of a particularly bad search tree because it is not really very tree-like at all. In fact, it is topologically isomorphic with a linear, linked list. In the worst case, a tree which contains n items has height O(n). Therefore, in the worst case an unsuccessful search must visit O(n) internal nodes.

figure18879
Figure: Examples of Search Trees

On the other hand, tree [image] in Figure gif is an example of a particularly good binary search tree. This tree is an instance of a perfect binary tree .

Definition (Perfect Binary Tree) A perfect binary tree of height [image] is a binary tree [image] with the following properties:
  1. If h=0, then [image] and [image].
  2. Otherwise, h>0, in which case both [image] and [image] are both perfect binary trees of height h-1.

It is fairly easy to show that a perfect binary tree of height h has exactly [image] internal nodes. Conversely, the height of a perfect binary tree with n internal nodes is [image]. If we have a search tree that has the shape of a perfect binary tree, then every unsuccessful search visits exactly h+1 internal nodes, where [image]. Thus, the worst case for unsuccessful search in a perfect tree is [image].


Average Case Analysis

  • Successful Search
  • Solving The Recurrence-Telescoping
  • Unsuccessful Search
  • Traversing a Search Tree


Successful Search

When a search is successful, exactly d+1 internal nodes are visited, where d is the depth in the tree of object of the search. E.g., if the object of the search is at the root which has depth zero, the search visits just one node--the root itself. Similarly, if the object of the search is at depth one, two nodes are visited, and so on. We shall assume that it is equally likely for the object of the search to appear in any node of the search tree. In that case, the average number of nodes visited during a successful search is [image], where [image] is the average of the depths of the nodes in a given tree. I.e., given a binary search tree with n>0 nodes,

where [image] is the depth of the [image] node of the tree.

The quantity [image] is called the internal path length . The internal path length of a tree is simply the sum of the depths (levels) of all the internal nodes in the tree. Clearly, the average depth of an internal node is equal to the internal path length divided by n, the number of nodes in the tree.

Unfortunately, for any given number of nodes n, there are many different possible search trees. Furthermore, the internal path lengths of the various possibilities are not equal. Therefore, to compute the average depth of a node in a tree with n nodes, we must consider all possible trees with n nodes. In the absence of any contrary information, we shall assume that all trees having n nodes are equiprobable and then compute the average depth of a node in the average tree containing n nodes.

Let I(n) be the average internal path length of a tree containing n nodes. Consider first the case of n=1. Clearly, there is only one binary tree that contains one node--the tree of height zero. Therefore, I(1)=0.

Now consider an arbitrary tree, [image], having [image] internal nodes altogether, l of which are found in its left subtree, where [image]. Such a tree consists of a root, the left subtree with l internal nodes and and a right subtree with n-l-1 internal nodes. The average internal path length for such a tree is the sum of the average internal path length of the left subtree, I(l), plus that of the right subtree, I(n-l-1), plus n-1 because the nodes in the two subtrees are one level lower in [image].

In order to determine the average internal path length for a tree with n nodes, we must compute the average of the internal path lengths of the trees [image] averaged over all possible sizes, l, of the (left) subtree, [image].

To do this we consider an ordered set of n distinct keys, [image]. If we select the [image] key, [image], to be the root of a binary search tree, then there are l keys, [image], [image], ..., [image], in its left subtree and n-l-1 keys, [image], [image], ..., [image] in its right subtree.

If we assume that it is equally likely for any of the n keys to be selected as the root, then all the subtree sizes in the range [image] are equally likely. Therefore, the average internal path length for a tree with [image] nodes is

Thus, in order to determine I(n) we need to solve the recurrence

To solve this recurrence we consider the case n>1 and then multiply Equation gif by n to get

Since this equation is valid for any n>1, by substituting n-1 for n we can also write

which is valid for n>2. Subtracting Equation gif from Equation gif gives

which can be rewritten as

Thus, we have shown the solution to the recurrence in Equation gif is the same as the solution of the recurrence


Solving The Recurrence-Telescoping

This section presents a technique for solving recurrence relations such as Equation gif called telescoping . The basic idea is this: We rewrite the recurrence formula so that a similar functional form appears on both sides of the equal sign. E.g., in this case, we consider n>2 and divide both sides of Equation gif by n+1 to get

Since this equation is valid for any n>2, we can write the following series of equations:

Each subsequent equation in this series is obtained by substituting n-1 for n in the preceding equation. In principle, we repeat this substitution until we get an expression on the right-hand-side involving the base case. In this example, we stop at n-k-1=2.

Because Equation gif has a similar functional form on both sides of the equal sign, when we add Equation gif through Equation gif together, most of the terms cancel leaving

where [image] is the [image] harmonic number . In Section gif it is shown that [image], where [image] is called Euler's constant . Thus, we get that the average internal path length of the average binary search tree with n internal nodes is

Finally, we get to the point: The average depth of a node in the average binary search tree with n nodes is


Unsuccessful Search

All successful searches terminate when the object of the search is found. Therefore, all successful searches terminate at an internal node. In contrast, all unsuccessful searches terminate at an external node. In terms of the binary tree shown in Figure gif , a successful search terminates in one of the nodes which are drawn as a circles and an unsuccessful search terminates in one of the boxes.

The preceding analysis shows that the average number of nodes visited during a successful search depends on the internal path length , which is simply the sum of the depths of all the internal nodes. Similarly, the average number of nodes visited during an unsuccessful search depends on the external path length , which is the sum of the depths of all the external nodes. Fortunately, there is a simple relationship between the internal path length and the external path length of a binary tree.

Theorem Consider a binary tree T with n internal nodes and an internal path length of I. The external path length of T is given by

In other words, Theorem gif says that the difference between the internal path length and the external path length of a binary tree with n internal nodes is E-I=2n.

extbfProof (By induction).

Base Case Consider a binary tree with one internal node and internal path length of zero. Such a tree has exactly two empty subtrees immediately below the root and its external path length is two. Therefore, the theorem holds for n=1.

Inductive Hypothesis Assume that the theorem holds for [image] for some [image]. Consider an arbitrary tree, [image], that has k internal nodes. According to Theorem gif , [image] has k+1 external nodes. Let [image] and [image] be the internal and external path length of [image], respectively, According to the inductive hypothesis, [image].

Consider what happens when we create a new tree [image] by removing an external node from [image] and replacing it with an internal node that has two empty subtrees. Clearly, the resulting tree has k+1 internal nodes. Furthermore, suppose the external node we remove is at depth d. Then the internal path length of [image] is [image] and the external path length of [image] is [image].

The difference between the internal path length and the external path length of [image] is

Therefore, by induction on k, the difference between the internal path length and the external path length of a binary tree with n internal nodes is 2n for all [image].

Since the difference between the internal and external path lengths of any tree with n internal nodes is 2n, then we can say the same thing about the average internal and external path lengths averaged over all search trees. Therefore, E(n), the average external path length of a binary search tree is given by

A binary search tree with internal n nodes has n+1 external nodes. Thus, the average depth of an external node of a binary search tree with n internal nodes, [image], is given by

These very nice results are the raison d'être for binary search trees. What they say is that the average number of nodes visited during either a successful or an unsuccessful search in the average binary search tree having n nodes is [image]. We must remember, however, that these results are premised on the assumption that all possible search trees of n nodes are equiprobable. It is important to be aware that in practice this may not always be the case.


Abstract Data Types and the Class Hierarchy

Chapter gif introduces the notion of an abstract data type. All of the data structures discussed in this book are presented as instances of various abstract data types. Chapter gif also introduces the class hierarchy as well as the various related concepts such as iterators and visitors.


Traversing a Search Tree

In Section gif , the inorder traversal of a binary tree is defined as follows:

  1. Traverse the left subtree; and then
  2. visit the root; and then
  3. traverse the right subtree.
It should not come as a surprise that when an inorder traversal of a binary search tree is done, the nodes of the tree are visited in order!

In an inorder traversal the root of the tree is visited after the entire left subtree has been traversed and in a binary search tree everything in the left subtree is less than the root. Therefore, the root is visited only after all the keys less than the root have been visited.

Similarly, in an inorder traversal the root is visited before the right subtree is traversed and everything in the right subtree is greater than the root. Hence, the root is visited before all the keys greater than the root are visited. Therefore, by induction, the keys in the search tree are visited in order.

Inorder traversal is not defined for arbitrary N-ary trees--it is only defined for the case of N=2. Essentially this is because the nodes of N-ary trees contain only a single key. On the other hand, if a node of an M-way search tree has n subtrees, then it must contain n-1 keys, such that [image]. Therefore, we can define inorder traversal of an M-way tree as follows:

To traverse a node of an M-way tree having n subtrees,

Traverse [image]; and then
visit [image]; and then
traverse [image]; and then
visit [image]; and then
traverse [image]; and then
2n-2.
visit [image]; and then
2n-1.
traverse [image].


Implementing Search Trees

Since search trees are designed to support efficient searching, it is only appropriate that they be implemented as classes derived from the SearchableContainer abstract base class. Recall from Section gif that the searchable container interface includes the member functions Find, IsMember, Insert, and Withdraw.

figure19290
Figure: Object Class Hierarchy

Program gif declares the SearchTree abstract base class. The SearchTree is derived from the classes Tree and SearchableContainer. The Tree class encapsulates those interface elements which are common to trees and search trees. The common member functions include Key, Subtree, IsEmpty, IsLeaf, Height, and Degree, as well as all of the various traversal routines (see Section gif ).

program19310
Program: SearchTree Class Definition

In addition, two more member functions are defined--FindMin and FindMax. These functions are accessors. The function FindMin returns a reference to the object contained in the search tree having the smallest key. Similarly, the function FindMax returns a reference to the contained object having the largest key.

  • Binary Search Trees
  • Inserting Items in a Binary Search Tree
  • Removing Items from a Binary Search Tree


Binary Search Trees

The class BST declared in Program gif represents binary search trees. Since binary trees and binary search trees are topologically similar, the BST class is derived from the BinaryTree class given in Section gif . In addition, because it represents search trees, the BST class is also derived from the SearchTree class.

program19333
Program: BST Class Definition

As shown in Program gif the BST class inherits much of its functionality from the BinaryTree base class. However, several functions are overridden, including AttachKey, DetachKey, Left and Right. In addition a new function, Balance, has been added to the protected interface. The purpose of this function is to support the AVL class discussed in Section gif .

  • Member Variables
  • Find Member Function
  • FindMin Member Function


Member Variables

The BST class inherits the three member variables key, left, and right for the BinaryTree class. All three of them are pointers. The first is a pointer to an Object instance, and the latter two are pointers to BinaryTree instances. All three pointers are zero if the node represents the empty tree. Otherwise, the tree must have a root and two subtrees. Therefore, all three pointers are non-zero in an internal node.

Program gif defines the two member functions Left and Right which return the left and right subtrees, respectively, of a given binary search tree. Since the implementation extends the BinaryTree class, the left and right member variables are declared as pointers to BinaryTrees. However, in a binary search tree, the subtrees will be instances of the BST class. Therefore, these functions make use of the dynamiccast operator to return a reference of the correct type.

program19374
Program: BST Class Left and Right Member Function Definitions


Find Member Function

Program gif gives the code for the Find member function of the BST class. The Find function takes as its lone argument a const reference to an Object instance. The purpose of the routine is to search the tree for an object which matches the argument. If a match is found, Find returns a reference to the matching object. Otherwise, Find returns NullObject::Instance().

program19396
Program: BST Class Find and FindMin Member Function Definitions

The recursive Find member function starts its search at the root and descends one level in the tree for each recursive call. At each level at most one object comparison is made (line 5). The worst case running time for a search is

where [image] is the time to compare two objects and n is the number of internal nodes in the tree. The same asymptotic running time applies for both successful and unsuccessful searches.

The average running time for a successful search is [image], where [image] is the average depth of an internal node in a binary search tree. If [image], the average time of a successful search is [image].

The average running time for an unsuccessful search is [image], where [image] is the average depth of an external node in a binary search tree. If [image], the average time of an unsuccessful search is [image].


FindMin Member Function

Program gif also shows a recursive implementation of the FindMin member function of the BST class. It follows directly from the data ordering property of search trees that to find the node containing the smallest key in the tree, we start at the root and follow the chain of left subtrees until we get to the node that has an empty left subtree. The key in that node is the smallest in the tree. Notice that no object comparisons are necessary to identify the smallest key in the tree.

The running time analysis of the FindMin routine follows directly from that of the Find function. The worst case running time of FindMin is O(n) and the average running time is [image], where n is the number of internal nodes in the tree.


Inserting Items in a Binary Search Tree

The simplest way to insert an item into a binary search tree is to pretend that the item is already in the tree and then follow the path taken by the Find routine to determine where the item would be. Assuming that the item is not already in the tree, the search will be unsuccessful and will terminate an an external, empty node. That is precisely where the item to be inserted is placed!

  • Insert and AttachKey Member Functions


Insert and AttachKey Member Functions

The Insert member function of the BST class is defined in Program gif . This function takes as its lone argument a reference to the object instance which is to be inserted into the binary search tree. It is assumed in this implementation that duplicate keys are not permitted. I.e., all of the keys contained in the tree are unique.

program19430
Program: BST Class Insert, AttachKey and Balance Member Function Definitions

The Insert routine behaves like the Find routine until it arrives at an external, empty node. Once the empty node has been found, it is transformed into an internal node by calling the AttachKey function. AttachKey works as follows: A pointer to the object being inserted is saved in the key member variable and two new empty binary trees are attached to the node.

Notice that after the insertion is done, the function Balance is called. However, as shown in Program gif , the BST::Balance function does nothing. Section gif describes the class AVLTree which is derived from the BST class and which inherits the Insert function but overrides the Balance operation.

The asymptotic running time of the Insert member function is the same as that of Find for an unsuccessful search. I.e., in the worst case the running time is [image] and the average case running time is

where [image] is the average depth of an external node in a binary search tree with n internal nodes. When [image], the worst case running time is O(n) and the average case is [image].


Removing Items from a Binary Search Tree

When removing an item from a search tree, it is imperative that the tree which remains satisfies the data ordering criterion. If the item to be removed is in a leaf node, then it is fairly easy to remove that item from the tree since doing so does not disturb the relative order of any of the other items in the tree.

For example, consider the binary search tree shown in Figure gif (a). Suppose we wish to remove the node labeled 4. Since node 4 is a leaf, its subtrees are empty. When we remove it from the tree, the tree remains a valid search tree as shown in Figure gif (b).

figure19467
Figure: Removing a Leaf Node from a Binary Search Tree

To remove a non-leaf node, we move it down in the tree until it becomes a leaf node since a leaf node is easily deleted. To move a node down we swap it with another node which is further down in the tree. For example, consider the search tree shown in Figure gif (a). Node 1 is not a leaf since it has an empty left subtree but a non-empty right subtree. To remove node 1, we swap it with the smallest key in its right subtree, which in this case is node 2, Figure gif (b). Since node 1 is now a leaf, it is easily deleted. Notice that the resulting tree remains a valid search tree, as shown in Figure gif (c).

figure19701
Figure: Removing a Non-Leaf Node from a Binary Search Tree

To move a non-leaf node down in the tree, we either swap it with the smallest key in the right subtree or with the largest one in the left subtree. At least one such swap is always possible, since the node is a non-leaf and therefore at least one of its subtrees is non-empty. If after the swap, the node to be deleted is not a leaf, the we push it further down the tree with yet another swap. Eventually, the node must reach the bottom of the tree where it can be deleted.

  • Withdraw and DetachKey Member Functions


Withdraw and DetachKey Member Functions

Program gif gives the code for the Withdraw and DetachKey member functions of the BST class. The Withdraw function takes as its lone argument a reference to the Object instance to be removed from the tree. The algorithm first determines the location of the object to be removed and then removes it according to the method described above.

program19945
Program: BST Class Withdraw and DetachKey Member Function Definitions


Data Structures

Chapter gif covers stacks, queues and deques. Ordered lists and sorted lists are presented in Chapter gif . The concept of hashing is introduced in Chapter gif . This chapter also covers the design of hash functions for a number of different object types. Finally, hash tables and scatter tables are presented.

Trees and search trees are presented in Chapters gif and gif . Trees are one of the most important non-linear data structures. Chapter gif also covers the various tree traversals, including depth-first traversal and breadth-first traversal. Chapter gif presents priority queues and Chapter gif covers sets, multisets and partitions.

An essential element of the C++ run-time system is the pool of dynamically allocated storage. Chapter gif presents a number of different approaches for implementing storage pools, in the process illustrating the actual costs associated with dynamic storage allocation.


AVL Search Trees

The problem with binary search trees is that while the average running times for search, insertion, and withdrawal operations are all [image], any one operation is still O(n) in the worst case. This is so because we cannot say anything in general about the shape of the tree.

For example, consider the two binary search trees shown Figure gif . Both trees contain the same set of keys. The tree [image] is obtained by starting with an empty tree and inserting the keys in the following order

The tree [image] is obtained by starting with an empty tree and inserting the keys in this order

Clearly, [image] is a better tree search tree than [image]. In fact, since [image] is a perfect binary tree , its height is [image]. Therefore, all three operations, search, insertion, and withdrawal, have the same worst case asymptotic running time, [image].

The reason that [image] is better than [image] is that it is the more balanced tree. If we could ensure that the search trees we construct are balanced then the worst-case running time of search, insertion, and withdrawal, could be made logarithmic rather than linear. But under what conditions is a tree balanced?

If we say that a binary tree is balanced if the left and right subtrees of every node have the same height, then the only trees which are balanced are the perfect binary trees. A perfect binary tree of height h has exactly [image] internal nodes. Therefore, it is only possible to create perfect trees with n nodes for [image]. Clearly, this is an unsuitable balance condition because it is not possible to create a balanced tree for every n.

What are the characteristics of a good balance condition ?

  1. A good balance condition ensures that the height of a tree with n nodes is [image].
  2. A good balance condition can be maintained efficiently. I.e., the additional work necessary to balance the tree when an item is inserted or deleted is O(1).
Adelson-Velskii and Landis gif were the first to propose the following balance condition and show that it has the desired characteristics.

Definition (AVL Balance Condition) An empty binary tree is AVL balanced . A non-empty binary tree, [image], is AVL balanced if both [image] and [image] are AVL balanced and

where [image] is the height of [image] and [image] is the height of [image].

Clearly, all perfect binary trees are AVL balanced. What is not so clear is that heights of all trees that satisfy the AVL balance condition are logarithmic in the number of internal nodes.

Theorem The height, h, of an AVL balanced tree with n internal nodes satisfies

extbfProof The lower bound follows directly from Theorem gif . It is in fact true for all binary trees regardless of whether they are AVL balanced.

To determine the upper bound, we turn the problem around and ask the question, what is the minimum number of internal nodes in an AVL balanced tree of height h?

Let [image] represent an AVL balanced tree of height h which has the smallest possible number of internal nodes, say [image]. Clearly, [image] must have at least one subtree of height h-1 and that subtree must be [image]. To remain AVL balanced, the other subtree can have height h-1 or h-2. Since we want the smallest number of internal nodes, it must be [image]. Therefore, the number of internal nodes in [image] is [image], where [image].

Clearly, [image] contains a single internal node, so [image]. Similarly, [image] contains exactly two nodes, so [image]. Thus, [image] is given by the recurrence

The remarkable thing about Equation gif is its similarity with the definition of Fibonacci numbers (Equation gif ). In fact, it can easily be shown by induction that

for all [image], where [image] is the [image] Fibonacci number.

Base Cases

Inductive Hypothesis Assume that [image] for [image]. Then

Therefore, by induction on k, [image], for all [image].

According to Theorem gif , the Fibonacci numbers are given by

where [image] and [image]. Furthermore, since [image], [image].

Therefore,

This completes the proof of the upper bound.

So, we have shown that the AVL balance condition satisfies the first criterion of a good balance condition--the height of an AVL balanced tree with n internal nodes is [image]. What remains to be shown is that the balance condition can be efficiently maintained. To see that it can, we need to look at an implementation.

  • Implementing AVL Trees
  • Inserting Items into an AVL Tree
  • Removing Items from an AVL Tree


Implementing AVL Trees

Having already implemented a binary search tree class, BST, we can make use of much of the existing code to implement an AVL tree class. Program gif gives the declaration of the AVLTree class which is derived from the class BST. The AVLTree class inherits most of its functionality from the binary tree class. In particular, it uses the inherited Insert and Withdraw functions! In addition, the inherited Balance, AttachKey and DetachKey member functions are overridden and a number of new member functions are declared.

program20048
Program: AVLTree Class Definition

Program gif indicates that the Height member function is redefined for the AVLTree class. This turns out to be necessary because we need to be able to determine quickly, i.e., in O(1) time, that the AVL balance condition is satisfied at a given node in the tree. In general, the running time required to compute the height of a tree containing n nodes is O(n). Therefore, to determine whether the AVL balance condition is satisfied at a given node, it is necessary to traverse completely the subtrees of the given node. But this cannot be done in constant time.

To make it possible to verify the AVL balance condition in constant time, the member variable height has been added. Thus, every node in an AVLTree keeps track of its own height. In this way it is possible for the Height member function to run in constant time--all it needs to do is to return the value of the height member variable. And this makes it possible to test whether the AVL balanced condition satisfied at a given node in constant time.

  • Constructor
  • Height, AdjustHeight and BalanceFactor Member Functions


Constructor

The default constructor for the AVLTree class is shown in Program gif . The default constructor creates an empty AVL tree. It does this by calling the default constructor of the BST class. The height field is set to the value -1, which is consistent with the empty tree. Notice that according to Definition gif , the empty tree is AVL balanced. Therefore, the result is a valid AVL tree. Clearly, the running time of the constructor is O(1).

program20094
Program: AVLTree Class Constructor, Height, AdjustHeight and BalanceFactor Member Function Definitions


Height, AdjustHeight and BalanceFactor Member Functions

The Height member function is implemented as an AVLTree member variable accessor that simply returns the value of the height member variable. Clearly the running time of this function is constant.

The purpose of AdjustHeight is to recompute the height of a node and to update the height member variable. This routine must be called whenever the height of one of the subtrees changes in order to ensure the height variable is always up to date. The AdjustHeight routine determines the height of a node by adding one to the height of the highest subtree. Since the running time of the Height function is constant, so too is the running time of AdjustHeight.

The BalanceFactor member function simply returns the difference between the heights of the left and right subtrees of a given AVL tree. By Definition gif , the empty node is AVL balanced. Therefore, the BalanceFactor routines zero for an empty tree. Again, since the running time of the Height function is constant, the running time of BalanceFactor is also constant.


Inserting Items into an AVL Tree

Inserting an item into an AVL tree is a two-part process. First, the item is inserted into the tree using the usual method for insertion in binary search trees. After the item has been inserted, it is necessary to check that the resulting tree is still AVL balanced and to balance the tree when it is not.

Just as in a regular binary search tree, items are inserted into AVL trees by attaching them to the leaves. To find the correct leaf we pretend that the item is already in the tree and follow the path taken by the Find routine to determine where the item should go. Assuming that the item is not already in the tree, the search is unsuccessful and terminates an an external, empty node. The item to be inserted is placed in that external node.

Inserting an item in a given external node affects potentially the heights of all of the nodes along the access path , i.e., the path from the root to that node. Of course, when an item is inserted in a tree, the height of the tree may increase by one. Therefore, to ensure that it resulting tree is still AVL balanced, the heights of all the nodes along the access path must be recomputed and the AVL balance condition must be checked.

Sometimes increasing the height of a subtree does not violate the AVL balance condition. For example, consider an AVL tree [image]. Let [image] and [image] be the heights of [image] and [image], respectively. Since T is an AVL tree, then [image]. Now, suppose that [image]. Then, if we insert an item into [image], its height may increase by one to [image]. The resulting tree is still AVL balanced since [image]. In fact, this particular insertion actually makes the tree more balanced! Similarly if [image] initially, an insertion in either subtree will not result in a violation of the balance condition at the root of T.

On the other hand, if [image] and an the insertion of an item into the left subtree [image] increases the height of that tree to [image], the AVL balance condition is no longer satisfied because [image]. Therefore it is necessary to change the structure of the tree to bring it back into balance.

  • Balancing AVL Trees
  • Single Rotations
  • Double Rotations
  • Implementation


Balancing AVL Trees

When an AVL tree becomes unbalanced, it is possible to bring it back into balance by performing an operation called a rotation . It turns out that there are only four cases to consider and each case has its own rotation.


Single Rotations

Figure gif (a) shows an AVL balanced tree. E.g., the balance factor for node A is zero, since its left and right subtrees have the same height; and the balance factor of node B is +1, since its left subtree has height h+1 and its right subtree has height h.

figure20135
Figure: Balancing an AVL Tree with a Single (LL) Rotation

Suppose we insert an item into [image], the left subtree of A. The height of [image] can either increase or remain the same. In this case we assume that it increases. Then, as shown in Figure gif (b), the resulting tree is no longer AVL balanced. Notice where the imbalance has been manifested--node A is balanced but node B is not.

Balance can be restored by reorganizing the two nodes A and B, and the three subtrees, [image], [image], and [image], as shown in Figure gif (c). This is called an LL rotation , because the first two edges in the insertion path from node B both go to the left.

There are three important properties of the LL rotation:

  1. The rotation does not destroy the data ordering property so the result is still a valid search tree. Subtree [image] remains to the left of node A, subtree [image] remains between nodes A and B, and subtree [image] remains to the right of node B.
  2. After the rotation both A and B are AVL balanced. Both nodes A and B end up with zero balance factors.
  3. After the rotation, the tree has the same height it had originally. Inserting the item did not increase the overall height of the tree!
Notice, the LL rotation was called for because the root became unbalanced with a positive balance factor (i.e., its left subtree was too high) and the left subtree of the root also had a positive balance factor.

Not surprisingly, the left-right mirror image of the LL rotation is called an RR rotation . An RR rotation is called for when the root becomes unbalanced with a negative balance factor (i.e., its right subtree is too high) and the right subtree of the root also has a negative balance factor.


Double Rotations

The preceding cases have dealt with access paths LL and RR. Clearly two more cases remain to be implemented. Consider the case where the root becomes unbalanced with a positive balance factor but the left subtree of the root has a negative balance factor. This situation is shown in Figure gif (b).

figure20532
Figure: Balancing an AVL Tree with a Double (LR) Rotation

The tree can be restored by performing an RR rotation at node A, followed by an LL rotation at node C. The tree which results is shown in Figure gif (c). The LL and RR rotations are called single rotations . The combination of the two single rotations is called a double rotation and is given the name LR rotation because the first two edges in the insertion path from node C both go left and then right.

Obviously, the left-right mirror image of the LR rotation is called an RL rotation . An RL rotation is called for when the root becomes unbalanced with a negative balance factor but the right subtree of the root has a positive balance factor. Double rotations have the same properties as the single rotations: The result is a valid, AVL-balanced search tree and the height of the result is the same as that of the initial tree.

Clearly the four rotations, LL, RR, LR, and RL, cover all the possible ways in which any one node can become unbalanced. But how many rotations are required to balance a tree when an insertion is done? The following theorem addresses this question:

Theorem When an AVL tree becomes unbalanced after an insertion, exactly one single or double rotation is required to balance the tree.

extbfProof When an item, x, is inserted into an AVL tree, T, that item is placed in an external node of the tree. The only nodes in T whose heights may be affected by the insertion of x are those nodes which lie on the access path from the root of T to x. Therefore, the only nodes at which an imbalance can appear are those along the access path. Furthermore, when a node is inserted into a tree, either the height of the tree remains the same or the height of the tree increases by one.

Consider some node c along the access path from the root of T to x. When x is inserted, the height of c either increases by one, or remains the same. If the height of c does not change, then no rotation is necessary at c or at any node above c in the access path.

If the height of c increases then there are two possibilities: Either c remains balanced or an imbalance appears at c. If c remains balanced, then no rotation is necessary at c. However, a rotation may be needed somewhere above c along the access path.

On the other hand, if c becomes unbalanced, then a single or a double rotation must be performed at c. After the rotation is done, the height of c is the same as it was before the insertion. Therefore, no further rotation is needed above c in the access path.

Theorem gif suggests the following method for balancing an AVL tree after an insertion: Begin at the node containing the item which was just inserted and move back along the access path toward the root. For each node determine its height and check the balance condition. If the height of the current node does not increase, then the tree is AVL balanced and no further nodes need be considered. If the node has become unbalanced, a rotation is needed to balance it. After the rotation, the height of the node remains unchanged, the tree is AVL balanced and no further nodes need be considered. Otherwise, the height of the node increases by one, but no rotation is needed and we proceed to the next node on the access path.


Implementation

Program gif gives the code for the LLRotation procedure of the AVLTree class. This code implements the LL rotation shown in Figure gif . The purpose of the LLRotation member function is to perform an LL rotation at the root of a given AVL tree instance.

program21061
Program: AVLTree Class LLRotation Member Function Definition

The rotation is simply a sequence of pointer manipulations followed by two height adjustments. Notice the rotation is done in such a way so that the the given AVLTree instance remains the root of the tree. This is done so that if the tree has a parent, it is not necessary to modify the contents of the parent.

The AVLTree class also requires an RRRotation member function to implement an RR rotation. The implementation of that function follows directly from Program gif . Clearly, the running time for the single rotations is O(1).

Program gif gives the implementation for the LRRotation member function of the AVLTree class. This double rotation is trivially implemented as a sequence of two single rotations. As above, the routine for the complementary rotation is easily derived from the given code. The running time for each of the double rotation functions is also O(1).

program21077
Program: AVLTree Class LRRotation Member Function Definition

When an imbalance is detected, it is necessary to correct the imbalance by doing the appropriate rotation. The code given in Program gif takes care of this. The Balance routine tests for an imbalance by calling the BalanceFactor function. The balance test itself takes constant time. If the node is balanced, only a constant-time height adjustment is needed.

program21089
Program: AVLTree Class Balance Member Function Definition

Otherwise, the Balance routine of the AVLTree class determines which of the four cases has occurred, and invokes the appropriate rotation to correct the imbalance. To determine which case has occurred, the Balance routine calls the BalanceFactor function two more times. Therefore, the time for selecting the case is constant. In all only one rotation is done to correct the imbalance. Therefore, the running time of this routine is O(1).

The Insert routine for AVL trees is inherited from the BST class (see Program gif ). The very last thing that routine does is to call the Balance function. which has been overridden. As a result the Insert routine adjusts the heights of the nodes along the insertion path and does a rotation when an imbalance is detected. Since the height of an AVL tree is guaranteed to be [image], the time for insertion is simply [image].


Removing Items from an AVL Tree

The procedure for removing items from an AVL tree is inherited from the BST class in the same way as AVL insertion. (See Program gif ). All the differences are encapsulated in the DetachKey and Balance functions. The Balance function is discussed above. The DetachKey function is defined in Program gif

program21115
Program: AVLTree Class AttachKey and DetachKey Member Function Definitions


Algorithms

The last three chapters of the book focus on algorithms, rather than data structures. Chapter gif is an overview of various algorithmic patterns. By introducing the notion of an abstract problem solver, we show how many of the patterns are related. Chapter gif uses a similar approach to present various sorting algorithms. I.e., we introduce the notion of an abstract sorter and show how the various sorting algorithms are related.

Finally, Chapter gif gives a brief overview of the subject of graphs an graph algorithms. This chapter brings together various algorithmic techniques from Chapter gif with the class hierarchy discussed in the earlier chapters.


M-Way Search Trees

As defined in Section gif , an internal node of an M-way search tree contains n subtrees and n-1 keys, where [image], for some fixed value of [image]. The preceding sections give implementations for the special case in which the fixed value of M=2 is assumed (binary search trees). In this section, we consider the implementation of M-way search trees for arbitrary, larger values of [image].

Why are we interested in larger values of M? Suppose we have a very large data set--so large that we cannot get it all into the main memory of the computer at the same time. In this situation we implement the search tree in secondary storage, i.e., on disk. The unique characteristics of disk-based storage vis-à-vis memory-based storage make it necessary to use larger values of M in order to implement search trees efficiently.

The typical disk access time is 1-10 ms, whereas the typical main memory access time is 10-100 ns. Thus, main memory accesses are between 10000 and 1000000 times faster than typical disk accesses. Therefore to maximize performance, it is imperative that the total number of disk accesses be minimized.

In addition, disks are block-oriented devices. Data are transfered between main memory and disk in large blocks. The typical block sizes are between 512 bytes and 4096 bytes. Consequently, it makes sense to organize the data structure to take advantage of the ability to transfer entire blocks of data efficiently.

By choosing a suitably large value for M, we can arrange that one node of an M-way search tree occupies an entire disk block. If every internal node in the M-way search tree has exactly M children, we can use Theorem gif to determine the height of the tree:

where n is the number of internal nodes in the search tree. A node in an M-way search tree that has M children contains exactly M-1 keys. Therefore, altogether there are K=(M-1)n keys and Equation gif becomes [image]. Ideally the search tree is well balanced and the inequality becomes an equality.

For example, consider a search tree which contains [image] keys. Suppose the size of a disk block is such that we can fit a node of size M=128 in it. Since each node contains at most 127 keys, at least 16513 nodes are required. In the best case, the height of the M-way search tree is only two and at most three disk accesses are required to retrieve any key! This is a significant improvement over a binary tree, the height of which is at least 20.

  • Implementing M-Way Search Trees
  • Finding Items in an M-Way Search Tree
  • Inserting Items into an M-Way Search Tree
  • Removing Items from an M-Way Search Tree


Implementing M-Way Search Trees

In order to illustrate the basic ideas, this section describes an implementation of M-way search trees in main memory. According to Definition gif , each internal node of an M-way search tree has n subtrees, where n is at least two and at most M. Furthermore, if a node has n subtrees, it must contain n-1 keys.

Figure gif shows how we can implement a single node of an M-way search tree. The idea is that we use two arrays in each node--the first holds the keys and the second contains pointers to the subtrees. Since there are at most M-1 keys and M subtrees, the first array is one element shorter than the second array.

figure21140
Figure: Representing a Node of an M-Way Search Tree

  • Implementation
  • Member Functions
  • Inorder Traversal


Implementation

Program gif declares the class MWayTree. An MWayTree is derived from the class SearchTree which is in turn derived from Tree and SearchableContainer. The four member variables, m, numberOfKeys, key, and subtree, correspond to the components of a node shown in Figure gif .

program21512
Program: MWayTree Class Definition

The first member variable, m, is a constant. It is used to record the degree of the node which cannot change after the node is created. (I.e., [image]). The second member variable, numberOfKeys, keeps track of the number of keys contained in the node. Recall, a node which has n subtrees contains n-1 keys. Therefore, [image]. We have chosen to keep track of the number of keys of a node rather than the number of subtrees because it simplifies the coding of the algorithms by eliminating some of the special cases which arise if we keep track of n explicitly.

The third member variable, key, is an array of pointers to Object class instances. It is used to contain pointers to the keys contained in the node. The fourth and final member variable, subtree, is an array of pointers to the MWayTree instances which are the subtrees of the given node.


Member Functions

Program gif also declares the constructor, the destructor, and the two member functions Key and Subtree. The constructor takes a single unsigned int argument, say M, and initializes the fields of the object as follows: The constant member variable m is set to M; the member variable numberOfKeys is set to zero; the key array is initialized to length M-1 and base 1; and, the subtree array is initialized to length M.

The member functions Key and Subtree are member variable accessors which are used to access the components of an MWayTree. The first, Key, takes an unsigned int argument between 1 and M-1 and returns a reference to the specified element of the key array. Similarly, the Subtree function takes an unsigned int argument between 0 and M-1 and returns a reference to the specified subtree.


Inorder Traversal

Whereas inorder traversal of an N-ary tree is not defined for N>2, inorder traversal is defined for an M-way search tree: By definition, the inorder traversal of a search tree visits all the keys contained in the search tree in order.

Program gif is an implementation of the algorithm for depth-first traversal of an M-way search tree given in Section gif . The keys contained in a given node are visited (by calling the Visit member function of the visitor) in between the appropriate subtrees of that node. In addition, the PreVisit function is called for each key contained in a node before traversing any of the subtrees of that node and the PostVisit function is called for each key in the node after all the subtrees have been traversed.

program21563
Program: MWayTree Class DepthFirstTraversal Member Function Definition

It is clear that the amount of work done at each node during the course of a depth-first traversal is proportional to the number of keys contained in that node. Therefore, the total running time for the depth-first traversal is [image], where K is the number of keys contained in the search tree.


Finding Items in an M-Way Search Tree

Two algorithms for finding items in an M-way search tree are described in this section. The first is a naıve implementation using linear search. The second version improves upon the first by using a binary search.

  • Linear Search
  • Binary Search


Linear Search

Program gif gives the naıve version of the Find member function of the MWayTree class. The Find member function takes a const reference to an Object instance and locates the item in the search tree which matches the given instance.

program21582
Program: MWayTree Class Find Member Function Definition(Linear Search)

Consider the execution of the Find function for a node T of a an M-way search tree. Suppose the object of the search is x. Clearly, the search fails when [image] (lines 3-4). In this case, a reference to the NullObject instance is returned. Suppose [image]. The linear search on lines 5-14 considers the keys [image], [image], [image], ..., [image], in that order. If a match is found, a reference to the matching object is returned immediately (lines 9-10).

Otherwise, when the main loop terminates there are three possibilities: i=0 and [image]; [image] and [image]; or i=n-1 and [image]. In all three cases, the appropriate subtree in which to continue search is [image] (line 15).

Clearly the running time of Program gif is determined by the main loop. In the worst case, the loop is executed M-1 times. Therefore, at each node in the search path at most M-1 object comparisons are done.

Consider an unsuccessful search in an M-way search tree. The running time of the Find function is

in the worst case, where h is the height of the tree and [image] is the time required to compare two objects. Clearly, the time for a successful search has the same asymptotic bound. If the tree is balanced and [image], then the running time of Program gif is [image], where K is the number of keys in the tree.


Binary Search

We can improve the performance of the M-way search tree search algorithm by recognizing that since the keys are kept in a sorted array, we can do a binary search rather than a linear search. Program gif gives an alternate implementation for the Find member function of the MWayTree class. This routine makes use of the private member function FindIndex which does the actual binary search.

program21608
Program: MWayTree Class FindIndex and Find Member Function Definitions (Binary Search)

The FindIndex member function takes as its lone argument a const reference to an Object instance, say x, and returns an unsigned int in the range between 0 and n-1, where n is the number of subtrees of the given node. The result is the largest integer i, if it exists, such that [image] where [image] is the [image] key. Otherwise, it returns the value 0.

FindIndex determines its result by doing a binary search. In the worst case, [image] iterations of the main loop (lines 9-16) are required to determine the correct index. One object comparison is done before the loop (line 5) and one comparison is done in each loop iteration (line 12). Therefore, the running time of the FindIndex function is

If [image], this simplifies to [image].

The Find member function of the MWayTree class does the actual search. It calls FindIndex to determine largest integer i, if it exists, such that [image] where [image] is the [image] key (line 24). If it turns out that [image], then the search is done (lines 25-26). Otherwise, Find calls itself recursively to search subtree [image] (line 28).

Consider a search in an M-way search tree. The running time of the second version of Find is

where h is the height of the tree and regardless of whether the search is successful. If the tree is balanced and [image], then the running time of Program gif is simply [image], where K is the number of keys in the tree.


Inserting Items into an M-Way Search Tree

The routine for inserting items in an M-way search tree follows directly from the algorithm for insertion in a binary search tree given in Section gif . The added wrinkle in an M-way tree is that an internal node may contain between 1 and M-1 keys whereas an internal node in a binary tree must contain exactly one key.

Program gif gives the implementation of the Insert member function of the MWayTree class. This function takes as its lone argument a reference to the Object instance to be inserted into the search tree.

program21640
Program: MWayTree Class Insert Member Function Definition

The general procedure for insertion is to search for the item to be inserted and then to insert it at the point where the search terminates. If the search terminates at an external node, that node is transformed to an internal node of the form [image], where x is the key just inserted (lines 5-8).

If the search terminates at an internal node, we insert the new item into the sorted list of keys at the appropriate offset. Inserting the key x in the array of keys moves all the keys larger than x and the associated subtrees to the right one position (lines 17-22). The hole in the list of subtrees is filled with an empty tree (line 23).

The preceding section gives the running time for a search in an M-way search tree as

where h is the height of the tree. The additional time required to insert the item into the node once the correct node has been located is O(M). Therefore, the total running time for the Insert algorithm given in Program gif is


Removing Items from an M-Way Search Tree

The procedure for removing items from an M-way search tree follows directly from the algorithm for removing items from a binary search tree given in Section gif . The basic idea is that the item to be deleted is pushed down the tree from its initial position to a node from which it can be easily deleted. Clearly, items are easily deleted from leaf nodes. In addition, consider an internal node of an M-way search tree of the form

If both [image] and [image] are empty trees, then the key [image] can be deleted from T by removing both [image] and [image], say. If [image] is non-empty, [image] can be pushed down the tree by swapping it with the largest key in [image]; and if [image] is non-empty, [image] can be pushed down the tree by swapping it with the smallest key in [image].

Program gif gives the code for the Withdraw function of the MWayTree class. The general form of the algorithm follows that of the Withdraw routine for the BST class (Program gif ).

program21665
Program: MWayTree Class Withdraw Member Function Definition


Algorithm Analysis

What is an algorithm and why do we want to analyze one? An algorithm is ``a...step-by-step procedure for accomplishing some end.''[9] An algorithm can be given in many ways. For example, it can be written down in English (or French, or any other ``natural'' language). However, we are interested in algorithms which have been precisely specified using an appropriate mathematical formalism--such as a programming language.

Given such an expression of an algorithm, what can we do with it? Well, obviously we can run the program and observe its behavior. This is not likely to be very useful or informative in the general case. If we run a particular program on a particular computer with a particular set of inputs, then all know is the behavior of the program in a single instance. Such knowledge is anecdotal and we must be careful when drawing conclusions based upon anecdotal evidence.

In order to learn more about an algorithm, we can ``analyze'' it. By this we mean to study the specification of the algorithm and to draw conclusions about how the implementation of that algorithm--the program--will perform in general. But what can we analyze? We can

  • determine the running time of a program as a function of its inputs;
  • determine the total or maximum memory space needed for program data;
  • determine the total size of the program code;
  • determine whether the program correctly computes the desired result;
  • determine the complexity of the program--e.g., how easy is it to read, understand, and modify; and,
  • determine the robustness of the program--e.g., how well does it deal with unexpected or erroneous inputs?

In this text, we are concerned primarily with the running time. We also consider the memory space needed to execute the program. There are many factors that affect the running time of a program. Among these are the algorithm itself, the input data, and the computer system used to run the program. The performance of a computer is determined by

  • the hardware:
    • processor used (type and speed),
    • memory available (cache and RAM), and
    • disk available;
  • the programming language in which the algorithm is specified;
  • the language compiler/interpreter used; and
  • the computer operating system software.

A detailed analysis of the performance of a program which takes all of these factors into account is a very difficult and time-consuming undertaking. Furthermore, such an analysis is not likely to have lasting significance. The rapid pace of change in the underlying technologies means that results of such analyses are not likely to be applicable to the next generation of hardware and software.

In order to overcome this shortcoming, we devise a ``model'' of the behavior of a computer with the goals of simplifying the analysis while still producing meaningful results. The next section introduces the first in a series of such models.

  • A Detailed Model of the Computer
  • A Simplified Model of the Computer
  • Exercises
  • Projects


B-Trees

Just as AVL trees are balanced binary search trees, B-trees are balanced M-way search trees. gif By imposing a balance condition , the shape of an AVL tree is constrained in a way which guarantees that the search, insertion, and withdrawal operations are all [image], where n is the number of items in the tree. The shapes of B-Trees are constrained for the same reasons and with the same effect.

Definition (B-Tree) A B-Tree of order M is either the empty tree or it is an M-way search tree T with the following properties:
  1. The root of T has at least two subtrees and at most M subtrees.
  2. All internal nodes of T (other than its root) have between [image] and M subtrees.
  3. All external nodes of T are at the same level.

A B-tree of order one is clearly impossible. Hence, B-trees of order M are really only defined for [image]. However, in practice we expect that M is large for the same reasons that motivate M-way search trees--large databases in secondary storage.

Figure gif gives an example of a B-tree of order M=3. By Definition gif , the root of a B-tree of order three has either two or three subtrees and the internal nodes also have either two or three subtrees. Furthermore, all the external nodes, which are shown as small boxes in Figure gif , are at the same level.

figure21694
Figure: A B-Tree of Order 3

It turns out that the balance conditions imposed by Definition gif are good in the same sense as the AVL balance conditions. I.e., the balance condition guarantees that the height of B-trees is logarithmic in the number of keys in the tree and the time required for insertion and deletion operations remains proportional to the height of the tree even when balancing is required.

Theorem The minimum number of keys in a B-tree of order [image] and height [image] is [image].

extbfProof Clearly, a B-tree of height zero contains at least one node. Consider a B-tree order M and height h>0. By Definition gif , each internal node (except the root) has at least [image] subtrees. This implies the minimum number of keys contained in an internal node is [image]. The minimum number of keys a level zero is 1; at level one, [image]; at level two, [image]; at level three, [image]; and so on.

Therefore the minimum number of keys in a B-tree of height h>0 is given by the summation

A corollary of Theorem gif is that the height, h, of a B-tree containing n keys is given by

Thus, we have shown that a B-tree satisfies the first criterion of a good balance condition--the height of B-tree with n internal nodes is [image]. What remains to be shown is that the balance condition can be efficiently maintained during insertion and withdrawal operations. To see that it can, we need to look at an implementation.

  • Implementing B-Trees
  • Inserting Items into a B-Tree
  • Removing Items from a B-Tree


Implementing B-Trees

Having already implemented the M-way search tree class, MWayTree, we can make use of much the existing code to implement a B-tree class. Program gif gives the declaration of the BTree class which is derived from the class MWayTree. With the exception of the two functions which modify the tree, Insert and Withdraw, the AVLTree class inherits all its public functionality from the M-way tree class. Of course, the Insert and Withdraw member functions need to be redefined in order to ensure that every time tree is modified the tree which results is a B-tree.

program21903
Program: BTree Class Definition

  • Member Variables
  • Constructors
  • Private Member Functions


Member Variables

To simplify the implementation of the algorithms, the member variable parent has been added. The parent member variable is a pointer to the BTree node which is the parent of the given node. Whereas the array of pointers to the children of a node, i.e., the subtree member variable of the MWayTree class, allows an algorithm to move down the tree; the parent member variable admits movement up the tree. Since the root of a tree has no parent, the parent member variable of the root node contains the null pointer.


Constructors

Two constructors are declared in Program gif The first takes a single unsigned int argument M and creates an empty B-tree of order M. The second constructor takes two arguments, an unsigned int and a pointer to an BTree. The integer argument specifies the order, M, of the tree to be constructed. The second argument specifies the parent of the node.


Private Member Functions

In addition to the public member functions, seven private member functions are declared in Program gif . These are used to implement the Insert and Withdraw member functions and are explained below.


Inserting Items into a B-Tree

The algorithm for insertion into a B-Tree begins as do all the other search tree insertion algorithms: To insert item x, we begin at the root and conduct a search for it. Assuming the item is not already in the tree, the unsuccessful search will terminate at a leaf node. This is the point in the tree at which the x is inserted.

If the leaf node has fewer than M-1 keys in it, we simply insert the item in the leaf node and we are done. E.g., consider a leaf node with n<M subtrees and n-1 keys of the form

For every new key inserted in the node, a new subtree is required too. In this case because T is a leaf, all its subtrees are empty trees. Therefore, when we insert item x, we really insert the pair of items [image]. Suppose the key to be inserted falls between [image] and [image], i.e., [image]. When we insert the pair [image] into T we get the new leaf T' given by

What happens when the leaf is full? I.e., suppose we wish to insert the pair, [image] into a node T which already has M-1 keys. Inserting the pair in its correct position gives a result of the form

However, this is not a valid node in a B-tree of order M because it has M+1 subtrees and M keys. The solution is to split node T' in half as follows

Note, [image] is a valid B-tree node because it contains [image] subtrees and [image] keys. Similarly, [image] is a valid B-tree node because it contains [image] subtrees and [image] keys. Note that there is still a key left over, namely [image].

There are now two cases to consider--either T is the root or it is not. Suppose T is not the root. Where we once had the single node T, we now have the two nodes, [image] and [image], and the left-over key, [image]. This situation is resolved as follows: First, [image] replaces T in the parent of T. Next, we take the pair [image] and recursively insert it in the parent of T.

Figure gif illustrates this case for a B-tree of order three. Inserting the key 6 in the tree causes the leaf node to overflow. The leaf is split in two. The left half contains key 5; and the right, key 7; and key 6 is left over. The two halves are re-attached to the parent in the appropriate place with the left-over key between them.

figure21969
Figure: Inserting Items into a B-Tree (Insert 6)

If the parent node fills up, then it too is split and the two new nodes are inserted in the grandparent. This process may continue all the way up the tree to the root. What do we do when the root fills up? When the root fills, it is also split. However, since there is no parent into which to insert the two new children, a new root is inserted above the old root. The new root will contain exactly two subtrees and one key, as allowed by Definition gif .

Figure gif illustrates this case for a B-tree of order three. Inserting the key 3 in the tree causes the leaf node to overflow. Splitting the leaf and reattaching it causes the parent to overflow. Similarly, splitting the parent and reattaching it causes the grandparent to overflow but the grandparent is the root. The root is split and a new root is added above it.

figure22221
Figure: Inserting Items into a B-Tree (Insert 3)

Notice that the height of the B-tree only increases when the root node splits. Furthermore, when the root node splits, the two halves are both attached under the new root. Therefore, the external nodes all remain at the same depth, as required by Definition gif .

  • Implementation
  • Running Time Analysis


Implementation

Insertion in a B-tree is a two-pass process. The first pass moves down the tree from the root in order to locate the leaf in which the insertion is to begin. This part of the algorithm is quite similar to the Find routine given in Program gif . The second pass moves from the bottom of the tree back up to the root, splitting nodes and inserting them further up the tree as needed. Program gif gives the code for the first (downward) pass (member function Insert) and the Program gif gives the code for the second (upward) pass (member function InsertPair).

program22668
Program: BTree Class Insert Member Function Definition

In the implementation shown, the downward pass starts at the root node and descends the tree until it arrives at an external node. If the external node has no parent, it must be the root and, therefore, the tree is empty. In this case, the root becomes an internal node containing a single key and two empty subtrees (lines 7-10). Otherwise, we have arrived at an external node in a non-empty tree and the second pass begins by calling InsertPair to insert the pair [image] in the parent.

The upward pass of the insertion algorithm is done by the recursive InsertPair routine shown in Program gif . The InsertPair routine takes two arguments. The first, object, is a pointer to an Object instance and the second, child, is a pointer to a BTree instance. It is assumed that all the keys in child are strictly greater than object.

program22687
Program: BTree Class InsertPair Member Function Definition

The InsertPair routine calls FindIndex to determine the position in the array of keys at which object should be inserted (line 3). It then calls InsertKey to insert the given key at the specified position in the array of keys (line 4). In the event that the array of keys is already full, i.e., when it contains M-1 items, the InsertKey function returns the key which falls off the right end of the array. This is assigned to extraKey.

The InsertSubtree function does a similar insertion (line 5). I.e., it inserts the child B-tree at the specified position. If the array of subtrees is full, which occurs if it already contains M subtrees, the InsertSubtree function returns the tree which falls of the right end of the array. This is assigned to extraTree.

If the numberOfKeys is equal to M, the node has overflowed and it is necessary to balance the B-tree. On the other hand, if the numberOfKeys is less than M, there is nothing more to do (line 7).

If the node overflows and it is the root, then two new B-trees, left and right are created (lines 11-12). The first [image] keys and [image] subtrees of the given node are moved to the left tree by the AttachLeftHalfOf function (line 13); and the last [image] keys and [image] subtrees of the given node are moved to the right tree by the AttachRightHalfOf function (line 14). The left-over key is the one in the middle of the array, i.e., [image]. Finally, the root node is modified so that it contains the two new subtrees and the single left-over key (lines 15-18).

If the node overflows and it is not the root, then one new B-tree is created, right (line 23). The last [image] keys and [image] subtrees of the given node are moved to the left tree by the AttachRightHalfOf function (line 24); and the first [image] keys and [image] subtrees of the given node remain attached to the given node. Finally, the InsertPair routine calls itself recursively to insert the left-over key, [image], and the new B-tree, right, into the parent of the given node (line 25). It should now be clear why the parent member variable is needed.


Running Time Analysis

The running time of the downward pass of the insertion algorithm is identical to that of an unsuccessful search (assuming the item to be inserted is not already in the tree). I.e., for a B-tree of height h, the worst-case running time of the downward pass is

The second pass of the insertion algorithm does the insertion and balances the tree if necessary. In the worst case, all of the nodes in the insertion path up to the root need to be balanced. Each time the InsertPair routine is invoked, it calls FindIndex which has running time [image] in the worst case. The additional time required to balance a node is O(M). Therefore, the worst-case running time of the upward pass is

Therefore, the total running time for insertion is

According to Theorem gif , the height of a B-tree is [image], where n is the number of keys in the B-tree. If we assume that two keys can be compared in constant time, i.e., [image], then the running time for insertion in a B-tree is simply [image].


Removing Items from a B-Tree

The procedure for removing items from a B-tree is similar to the algorithm for removing item from an AVL tree. I.e., once the item to be removed has be found, it is pushed down the tree to a leaf node where it can be easily deleted. When an item is deleted from a node it is possible that the number of keys remaining is less than [image]. In this case, balancing is necessary.

The procedure of balancing after deletion is like the balancing after insertion in that it progresses from the leaf node up the tree toward the root. Given a node T which has [image] keys, there are four cases to consider.

In the first case, T is the root. If no keys remain, T becomes the empty tree. Otherwise, no balancing is needed because the root is permitted to have as few as two subtrees and one key. For the remaining cases T is not the root.

In the second case T has [image] keys and it also has a sibling immediately on the left with at least [image] keys. The tree can be balanced by doing an LL rotation as shown in Figure gif . Notice that after the rotation, both siblings have at least [image] keys. Furthermore, the heights of the siblings remain unchanged. Therefore, the resulting tree is a valid B-tree.

figure22731
Figure: LL Rotation in a B-Tree

The third case is the left-right mirror of the second case. I.e., T has [image] keys and it also has a sibling immediately on the right with a least [image] keys. In this case, the tree can be balanced by doing an RR rotation .

In the fourth and final case, T has [image] keys, and its immediate sibling(s) have [image] keys. In this case, the sibling(s) cannot give-up a key in a rotation because they already have the minimum number of keys. The solution is to merge T with one of its siblings as shown in Figure gif .

figure23218
Figure: Merging nodes in a B-Tree

The merged node contains [image] keys from T, [image] keys from the sibling, and one key from the parent (the key x in Figure gif ). The resulting node contains [image] keys altogether, which is M-2 if M is even and M-1 if M is odd. Either way, the resulting node contains no more than M-1 keys and is a valid B-tree node. Notice that in this case a key has been removed from the parent of T. Therefore, it may be necessary to balance the parent. Balancing the parent may necessitate balancing the grandparent, and so on, up the tree to the root.


Applications

There are many applications for search trees. The principal characteristic of such applications is that a database of keyed information needs to be frequently accessed and the access pattern is either unknown or known to be random. E.g., dictionaries are often implemented using search trees. A dictionary is essentially a container that contains ordered key/value pairs. The keys are words is a source language and, depending on the application, the values may be the definitions of the words or the translation of the word in a target language.

This section presents a simple application of search trees. Suppose we are required to translate the words in an input file one-by-one from some source language to another target language. In this example, the translation is done one word at a time. I.e., no natural language syntactic or semantic processing is done.

In order to implement the translator we assume that there exists a text file, say "dict.txt" which contains pairs of words. The first element of the pair is a word in the source language and the second element is a word in the target language. To translate a text, we first read the words and the associated translations and build a search tree. The translation is created one word at a time by looking up each word in the text.

Program gif gives an implementation of the translator. Two functions are given, ReadDictionary and TranslateText. The first function takes a reference to a SearchTree instance which is used to hold the pairs of words. Notice, this implementation works with all the search tree types described in this chapter (e.g., BST, AVLTree, MWayTree, and BTree).

program23633
Program: Application of Search Trees--Word Translation

The ReadDictionary reads pairs of strings from the file "dict.txt" (lines 6-7). The Association class defined in Section gif is used to contain the key/value pairs. A new instance is created for each key/value pair which is then inserted into the search tree (lines 8-9). The process of building the search tree terminates when the end-of-file is encountered.

The TranslateText function takes as its lone argument a reference to a search tree which is assumed to contain the set of key/value pairs read in by the ReadDictionary function. The TranslateText function reads words one at a time from the standard input, cin, and writes the translation of each word on the standard output, cout. Each word is looked up as it is read (lines 18-19). If no key matches the given word, the word is printed followed by a question mark (lines 20-21). Otherwise, the value associated with the matching key is printed (lines 24-26).


A Detailed Model of the Computer

In this section we develop a detailed model of the running time performance of C++ programs. The model developed is independent of the underlying hardware and system software. Rather than analyze the performance of a particular, arbitrarily chosen physical machine, we consider the implementation of the C++ programming language as a kind a ``virtual C++ machine'' (see Figure gif ).

A direct consequence of this approach is that we lose some fidelity--the resulting model will not be able to accurately predict the performance of all possible hardware/software systems. On the other hand, we shall see that the resulting model is still rather complex and rich in detail.

figure239
Figure: Virtual C++ Machine

  • The Basic Axioms
  • A Simple Example-Arithmetic Series Summation
  • Array Subscripting Operations
  • Another Example-Horner's Rule
  • Analyzing Recursive Functions
  • Yet Another Example-Finding the Largest Element of an Array
  • Average Running Times
  • About Harmonic Numbers
  • Best-Case and Worst-Case Running Times
  • The Last Axiom


Exercises

  1. For each of the following key sequences determine the binary search tree obtained when the keys are inserted one-by-one in the order given into an initially empty tree:
    1. 1, 2, 3, 4, 5, 6, 7.
    2. 4, 2, 1, 3, 6, 5, 7.
    3. 1, 6, 7, 2, 4, 3, 5.
  2. For each of the binary search trees obtained in Exercise gif determine the tree obtained when the root is withdrawn.
  3. Repeat Exercises gif and gif for AVL trees.
  4. Derive an expression for the total space needed to represent a tree of n internal nodes using each of the following classes:
    1. BST defined in Program gif ,
    2. AVLTree defined in Program gif ,
    3. MWayTree defined in Program gif , and
    4. BTree defined in Program gif .
    Hint: For the MWayTree and BTree assume that the tree contains are k keys, where [image].
  5. To delete a non-leaf node from a binary search tree, we swap it either with the smallest key its right subtree or with the largest key in its left subtree and then recursively delete it from the subtree. In a tree of n nodes, what its the maximum number of swaps needed to delete a key?
  6. Devise an algorithm to compute the internal path length of a tree. What is the running time of your algorithm?
  7. Devise an algorithm to compute the external path length of a tree. What is the running time of your algorithm?
  8. Suppose that you are given a sorted sequence of n keys, [image], to be inserted into a binary search tree.
    1. What is the minimum height of a binary tree that contains n nodes.
    2. Devise an algorithm to insert the given keys into a binary search tree so that the height of the resulting tree is minimized.
    3. What is the running time of your algorithm?
  9. Devise an algorithm to construct an AVL tree of a given height h that contains the minimum number of nodes. The tree should contain the keys [image], where [image] is given by Equation gif .
  10. Consider what happens when we insert the keys [image] one-by-one in the order given into an initially empty AVL tree for [image]. Prove that the result is always a perfect tree of height h.
  11. The Find routine defined in Program gif is recursive. Write a non-recursive routine to find a given item in a binary search tree.
  12. Repeat Exercise gif for the FindMin function defined in Program gif .
  13. Devise an algorithm to select the [image] key in a binary search tree. E.g., given a tree with n nodes, k=0 selects the smallest key, k=n-1 selects the largest key, and [image] selects the median key.
  14. Devise an algorithm to test whether a given binary search tree is AVL balanced. What is the running time of your algorithm?
  15. Devise an algorithm that takes two values, a and b such that [image], and which visits all the keys x in a binary search tree such that [image]. The running time of your algorithm should be [image], where N is the number of keys visited and n is the number of keys in the tree.
  16. Devise an algorithm to merge the contents of two binary search trees into one. What is the running time of your algorithm?
  17. (This question should be attempted after reading Chapter gif ). Prove that a complete binary tree (Definition gif ) is AVL balanced.
  18. Do Exercise gif .
  19. For each of the following key sequences determine the 3-way search tree obtained when the keys are inserted one-by-one in the order given into an initially empty tree:
    1. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
    2. 3, 1, 4, 5, 9, 2, 6, 8, 7, 0.
    3. 2, 7, 1, 8, 4, 5, 9, 0, 3, 6.
  20. Repeat Exercise gif for B-trees of order 3.


Projects

  1. Complete the implementation of the BST class declared in Program gif by providing suitable definitions for the following member functions: IsMember and FindMax. You must also have a complete implementation of the base class BinaryTree. (See Project gif ). Write a test program and test your implementation.
  2. Complete the implementation of the AVLTree class declared in Program gif by providing suitable definitions for the following member functions: Left, Right, RRRotation and RLRotation. You must also have a complete implementation of the base class BST. (See Project gif ). Write a test program and test your implementation.
  3. Complete the implementation of the MWayTree class declared in Program gif by providing suitable definitions for the following member functions: MWayTree (constructor), MWayTree (destructor), Purge, Count, IsEmpty, IsLeaf, Degree, Key, Subtree, IsMember, FindMin, FindMax, BreadthFirstTraversal and NewIterator. Write a test program and test your implementation.
  4. Complete the implementation of the BTree class declared in Program gif by providing suitable definitions for the following member functions: BTree, InsertKey, InsertSubtree, AttachKey, AttachSubtree, AttachLeftHalfOf, AttachRightHalfOf and Withdraw. You must also have a complete implementation of the base class MWayTree. (See Project gif ). Write a test program and test your implementation.
  5. The binary search tree Withdraw routine shown in Program gif is biased in the following way: If the key to be deleted is in a non-leaf node with two non-empty subtrees, the key is swapped with the maximum key in the left subtree and then recursively deleted from the left subtree. Following a long series of insertions and deletions, the search tree will tend to have more nodes in the right subtrees and fewer nodes in the left subtrees. Devise and conduct an experiment that demonstrates this phenomenon.
  6. Consider the implementation of AVL trees. In order to check the AVL balance condition in constant time, we record in each node the height of that node. An alternative to keeping track of the height information explicitly is to record in each node the difference in the heights of its two subtrees. In an AVL balanced tree, this difference is either -1, 0 or +1. Replace the height member variable of the AVL class defined in Program gif with one called diff and rewrite the various member functions accordingly.
  7. The M-way tree implementation given in Section gif is an internal data structure--it is assumed that all the nodes reside in the main memory. However, the motivation for using an M-way tree is that it is an efficient way to organize an external data structure--one that is stored on disk. Design, implement and test an external M-way tree implementation.


Heaps and Priority Queues

In this chapter we consider priority queues. A priority queue is essentially a list of items in which each item has associated with it a priority. In general, different items may have different priorities and we speak of one item having a higher priority than another. Given such a list we can determine which is the highest (or the lowest) priority item in the list. Items are inserted into a priority queue in any, arbitrary order. However, items are withdrawn from a priority queue in order of their priorities starting with the highest priority item first.

For example, consider the software which manages a printer. In general, it is possible for users to submit documents for printing much more quickly than it is possible to print them. A simple solution is to place the documents in a FIFO queue (Chapter gif ). In a sense this is fair, because the documents are printed on a first-come, first-served basis.

However, a user who has submitted a short document for printing will experience a long delay when much longer documents are already in the queue. An alternative solution is to use a priority queue in which the shorter a document, the higher its priority. By printing the shortest documents first, we reduce the level of frustration experienced by the users. In fact, it can be shown that printing documents in order of their length minimizes the average time a user waits for her document.

Priority queues are often used in the implementation of algorithms. Typically the problem to be solved consists of a number of subtasks and the solution strategy involves prioritizing the subtasks and then performing those subtasks in the order of their priorities. For example, in Chapter gif we show how a priority queue can improve the performance of backtracking algorithms, in Chapter gif we will see how a priority queue can be used in sorting and in Chapter gif several graph algorithms that use a priority queue are discussed.

  • Basics
  • Binary Heaps
  • Leftist Heaps
  • Binomial Queues
  • Applications
  • Exercises
  • Projects


Basics

A priority queue is a container which provides the following three operations:

Enqueue
used to put objects into the container;
FindMin
returns a reference to the smallest object in the container; and
DequeueMin
removes the smallest object from the container.

A priority queue is used to store a finite set of keys drawn from a totally ordered set of keys K. As distinct from search trees, duplicate keys are allowed in priority queues.

Program gif gives the declaration of the PriorityQueue abstract class. The PriorityQueue class is derived from the Container class. In addition to the inherited functions, the public interface of the PriorityQueue class comprises the three functions listed above.

program23974
Program: PriorityQueue and MergeablePriorityQueue Class Definitions

Program gif also declares one additional class--MergeablePriorityQueue. A mergeable priority queue is one which provides the ability to merge efficiently two priority queues into one. Of course it is always possible to merge two priority queues by dequeuing the elements of one queue and enqueuing them in the other. However, the mergeable priority queue implementations we will consider allow more efficient merging than this.

figure23996
Figure: Object Class Hierarchy

It is possible to implement the required functionality using data structures that we have already considered. For example, a priority queue can be implemented simply as a list. If an unsorted list is used, enqueuing can be accomplished in constant time. However, finding the minimum and removing the minimum each require O(n) time where n is the number of items in the queue. On the other hand, if an sorted list is used, finding the minimum and removing it is easy--both operations can be done in constant time. However, enqueuing an item in an sorted list requires O(n) time.

Another possibility is to use a search tree. For example, if an AVL tree is used to implement a priority queue, then all three operations can be done in [image] time. However, search trees provide more functionality than we need. Viz., search trees support finding the largest item with FindMax, deletion of arbitrary objects with Withdraw, and the ability to visit in order all the contained objects via DepthFirstTraversal. All these operations can be done as efficiently as the priority queue operations. Because search trees support more functions than we really need for priority queues, it is reasonable to suspect that there are more efficient ways to implement priority queues. And indeed there are!

A number of different priority queue implementations are described in this chapter. All the implementations have one thing in common--they are all based on a special kind of tree called a min heap or simply a heap.

Definition ((Min) Heap) A (Min) Heap is a tree,

with the following properties:

  1. Every subtree of T is a heap; and,
  2. The root of T is less than or equal to the root of every subtree of T. I.e., [image], where [image] is the root of [image].
According to Definition gif , the key in each node of a heap is less than or equal to the roots of all the subtrees of that node. Therefore, by induction, the key in each node is less than or equal to all the keys contained in the subtrees of that node. Note, however, that the definition says nothing about the relative ordering of the keys in the subtrees of a given node. For example, in a binary heap either the left or the right subtree of a given node may have the larger key.


Binary Heaps

A binary heap is a heap-ordered binary tree which has a very special shape called a complete tree. As a result of its special shape, a binary heap can be implemented using an array as the underlying foundational data structure. Thus, the implementation is based on array subscript calculations rather than pointer manipulations. And since an array is used, the storage overhead associated with the pointers contained in the nodes of the trees is eliminated.

  • Complete Trees
  • Implementation
  • Putting Items into a Binary Heap
  • Removing Items from a Binary Heap


Complete Trees

The preceding chapter introduces the idea of a perfect tree (see Definition gif ). Complete trees and perfect trees are closely related, yet quite distinct. As pointed out in the preceding chapter, a perfect binary tree of height h has exactly [image] internal nodes. Since, the only permissible values of n are

there is no perfect binary tree which contains, say 2, 4, 5, or 6 nodes.

However, we want a data structure that can hold an arbitrary number of objects so we cannot use a perfect binary tree. Instead, we use a complete binary tree, which is defined as follows:

Definition (Complete Binary Tree) A complete binary tree of height [image], is a binary tree [image] with the following properties.
  1. If h=0, [image] and [image].
  2. For h>0 there are two possibilities:
    1. is a perfect binary tree of height h-1 and [image] is a complete binary tree of height h-1; or
    2. is a complete binary tree of height h-1 and [image] is a perfect binary tree of height h-2.

Figure gif shows an example of a complete binary tree of height four. Notice that the left subtree of node 1 is a complete binary tree of height three; and the right subtree is a perfect binary tree of height two. This corresponds to case 2 (b) of Definition gif . Similarly, the left subtree of node 2 is a perfect binary tree of height two; and the right subtree is a complete binary tree of height two. This corresponds to case 2 (a) of Definition gif .

figure24050
Figure: A Complete Binary Tree

Does there exist an complete binary with exactly n nodes for every integer n>0? The following theorem addresses this question indirectly by defining the relationship between the height of a complete tree and the number of nodes it contains.

Theorem A complete binary tree of height [image] contains at least [image] and at most [image] nodes.

extbfProof First, we prove the lower bound by induction. Let [image] be the minimum number of nodes in a complete binary tree of height h. To prove the lower bound we must show that [image].

Base Case There is exactly one node in a tree of height zero. Therefore, [image].

Inductive Hypothesis Assume that [image] for [image], for some [image]. Consider the complete binary tree of height k+1 which has the smallest number of nodes. Its left subtree is a complete tree of height k having the smallest number of nodes and its right subtree is a perfect tree of height k-1.

From the inductive hypothesis, there are [image] nodes in the left subtree and there are exactly [image] nodes in the perfect right subtree. Thus,

Therefore, by induction [image] for all [image], which proves the lower bound.

Next, we prove the upper bound by induction. Let [image] be the maximum number of nodes in a complete binary tree of height h. To prove the upper bound we must show that [image].

Base Case There is exactly one node in a tree of height zero. Therefore, [image].

Inductive Hypothesis Assume that [image] for [image], for some [image]. Consider the complete binary tree of height k+1 which has the largest number of nodes. Its left subtree is a perfect tree of height k and its right subtree is a complete tree of height k having the largest number of nodes.

There are exactly [image] nodes in the perfect left subtree. From the inductive hypothesis, there are [image] nodes in the right subtree. Thus,

Therefore, by induction [image] for all [image], which proves the upper bound.

It follows from Theorem gif that there exists exactly one complete binary tree that contains exactly n internal nodes for every integer [image]. It also follows from Theorem gif that the height of a complete binary tree containing n internal nodes is [image].

Why are we interested in complete trees? As it turns out, complete trees have some useful characteristics. For example, in the preceding chapter we saw that the internal path length of a tree, i.e., the sum of the depths of all the internal nodes, determines the average time for various operations. A complete binary tree has the nice property that it has the smallest possible internal path length:

Theorem The internal path length of a binary tree with n nodes is at least as big as the internal path length of a complete binary tree with n nodes.

extbfProof Consider a binary tree with n nodes that has the smallest possible internal path length. Clearly, there can only be one node at depth zero--the root. Similarly, at most two nodes can be at depth one; at most four nodes can be at depth two; and so on. Therefore, the internal path length of a tree with n nodes is always at least as large as the sum of the first n terms in the series

But this summation is precisely the internal path length of a complete binary tree!

Since the depth of the average node in a tree is obtained by dividing the internal path length of the tree by n, Theorem gif tells us that complete trees are the best possible in the sense that the average depth of a node in a complete tree is the smallest possible. But how small is small? I.e., is does the average depth grow logarithmically with n. The following theorem addresses this question:

Theorem The internal path length of a complete binary tree with n nodes is

extbfProof The proof of Theorem gif is left as an exercise for the reader (Exercise gif ).

From Theorem gif we may conclude that the internal path length of a complete tree is [image]. Consequently, the depth of the average node in a complete tree is [image].

  • Complete N-ary Trees


Complete N-ary Trees

The definition for complete binary trees can be easily extended to trees with arbitrary fixed degree [image] as follows:

Definition (Complete N-ary Tree) A complete N-ary tree of height [image], is an N-ary tree [image] with the following properties.
  1. If h=0, [image] for all i, [image].
  2. For h>0 there exists a j, [image] such that
    1. is a perfect binary tree of height h-1 for all [image];
    2. is a complete binary tree of height h-1; and,
    3. is a perfect binary tree of height h-2 for all i:j<i<N.

Note that while it is expressed in somewhat different terms, the definition of a complete N-ary tree is consistent with the definition of a binary tree for N=2. Figure gif shows an example of a complete ternary (N=3) tree.

figure24290
Figure: A Complete Ternary Tree

Informally, a complete tree is a tree in which all the levels are full except for the bottom level and the bottom level is filled from left to right. For example in Figure gif , the first three levels are full. The fourth level which comprises nodes 14-21 is partially full and has been filled from left to right.

The main advantage of using complete binary trees is that they can be easily stored in an array. Specifically, consider the nodes of a complete tree numbered consecutively in level-order as they are in Figures gif and gif . There is a simple formula that relates the number of a node with the number of its parent and the numbers of its children.

Consider the case of a complete binary tree. The root node is node 1 and its children are nodes 2 and 3. In general, the children of node i are 2i and 2i+1. Conversely, the parent of node i is [image]. Figure gif illustrates this idea by showing how the complete binary tree shown in Figure gif is mapped into an array. When using this approach, the pointers are no longer explicitly recorded.

figure24590
Figure: Array Representation of a Complete Binary Tree

A remarkable characteristic of complete trees is that filling the bottom level from left to right corresponds to adding elements at the end of the array! Thus, a complete tree containing n nodes occupies the first n consecutive array positions.

The array subscript calculations given above can be easily generalized to complete N-ary trees. Assuming that the root occupies position 1 of the array, its N children occupy positions 2, 3, ..., N+1. In general, the children of node i occupy positions

and the parent of node i is found at


Implementation

A binary heap is a heap-ordered complete binary tree which is implemented using an array. In a heap the smallest key is found at the root and since the root is always found in the first position of the array, finding the smallest key is a trivial operation in a binary heap.

Program gif declares the class BinaryHeap. The BinaryHeap class is derived from the PriorityQueue abstract base class. Since BinaryHeap is a concrete class it provides implementations for all of the member functions declared as pure virtual functions in the base class. In the interest of brevity, the declarations of these functions have been elided from Program gif .

program24847
Program: BinaryHeap Class Definition

  • Member Variables
  • Constructor, Destructor and Purge Member Functions


Member Variables

The BinaryHeap class has a rather simple implementation. In particular, it requires only a single member variable, array, which is declared as an array of pointers to Object class instances. This array is used to hold pointers to the objects which are contained in the binary tree. When there are n items in the heap, the pointers to those items occupy the first n array positions.


Constructor, Destructor and Purge Member Functions

Program gif defines the BinaryHeap class constructor. The constructor takes a single argument of type unsigned int which specifies the maximum capacity of the binary heap. The argument passed on to the constructor for the array member variable. Notice too that the array is initialized so that its positions are numbered starting from subscript 1 rather than the default of zero.

program24875
Program: BinaryHeap Class Constructor, Destructor and Purge Member Function Definitions

The purpose of the Purge member function of the BinaryHeap class is to delete from the priority queue all contained and owned objects. Pointers to the contained objects are found in the first [image] array positions. Clearly the worst-case running time for the Purge function is

where [image] is the worst-case time required to delete an object instance.

The implementation of the destructor for the BinaryHeap class is trivial as it simply calls Purge to do its work. Consequently, the destructor has the same running time as above.


The Basic Axioms

The running time performance of the C++ virtual machine is given by a set of axioms which we shall now postulate. For now we consider only operations on integers. The first axiom addresses the running time of simple variable references:

Axiom The time required to fetch an integer operand from memory is a constant, [image], and the time required to store an integer result in memory is a constant, [image].

According to Axiom gif , the assignment statement

y = x;
has running time [image]. I.e., the time taken to fetch the value of variable x is [image] and the time taken to store the value in variable y is [image].

We shall apply Axiom gif to manifest constants too: The assignment

y = 1;
also has running time [image]. To see why this should be the case, consider that the constant typically needs to be stored in the memory of the computer, and we can expect the cost of fetching it to be the same as that of fetching any other operand.

The next axiom addresses the running time of simple arithmetic operations on integers:

Axiom The times required to perform elementary operations on integers, such as addition, subtraction, multiplication, division, and comparison, are all constants. These times are denoted by [image], [image], [image], [image], and [image], respectively.

According to Axiom gif , all the simple operations on integers can be accomplished in a fixed amount of time. In order for this to be feasible, the number of bits used to represent an integer must be fixed. Typically, between 16 and 64 bits are used to represent an integer. It is precisely because the number of bits used is fixed that we can say that the running times are also fixed. If arbitrarily large integers are allowed, then the basic arithmetic operations can take an arbitrarily long amount of time.

By applying Axioms gif and gif , we can determine that the running time of a statement like

y = y + 1;
is [image]. This is because we need to fetch two operands, y and 1; add them; and, store the result back in y.

C++ syntax provides several alternative ways to express the same computation:

y += 1;
++y;
y++;
We shall assume that these alternatives require exactly the same running time as the original statement.

The third basic axiom addresses the function call/return overhead:

Axiom The time required to call a function is a constant, [image], and the time required to return from a function is a constant, [image].

When a function is called, certain housekeeping operations need to be performed. Typically this includes saving the return address so that program execution can resume at the correct place after the call, saving the state of any partially completed computations so that they may be resumed after the call, and the allocation of a new execution context (stack frame or activation record ) in which the called function can be evaluated. Conversely, on the return from a function, all of this work is undone. While the function call/return overhead may be rather large, nevertheless it entails a constant amount of work.

In addition to the function call/return overhead, additional overhead is incurred when parameters are passed to the function:

Axiom The time required to pass an integer argument to a function or procedure is the same as the time required to store an integer in memory, [image].

The rationale for making the overhead associated with parameter passing the same as the time to store a value in memory is that the passing of an argument is conceptually the same as assignment of the actual parameter value to the formal parameter of the function.

According to Axiom gif , the running time of the statement

y = f (x);
would be [image], where [image] is the running time of function f for input x. The first of the two stores is due to the passing of the parameter x to the function f; the second arises from the assignment to the variable y.


Putting Items into a Binary Heap

There are two requirements which must be satisfied when an item is inserted in a binary heap. First, the resulting tree must have the correct shape. Second, the tree must remain heap-ordered. Figure gif illustrates the way in which this is done.

Since the resulting tree must be a complete tree, there is only one place in the tree where a node can be added. I.e., since the bottom level must be filled from left to right, the node node must be added at the next available position in the bottom level of the tree as shown in Figure gif (a).

figure24897
Figure: Inserting an Item into a Binary Heap

In this example, the new item to be inserted has the key 2. Note that we cannot simply drop the new item into the next position in the complete tree because the resulting tree is no longer heap ordered. Instead, the hole in the heap is moved toward the root by moving items down in the heap as shown in Figure gif (b) and (c). The process of moving items down terminates either when we reach the root of the tree or when the hole has been moved up to a position in which when the new item is inserted the result is a heap.

Program gif gives the code for inserting an item in a binary heap. The Enqueue member function of the BinaryHeap class takes as its lone argument a reference to the item to be inserted in the heap. If the priority queue is full an exception is thrown. Otherwise, the item is inserted as described above.

program25217
Program: BinaryHeap Class Enqueue Member Function Definition

The implementation of the algorithm is actually remarkably simple. Lines 6-11 move the hole in the heap up by moving items down. When the loop terminates, the new item can be inserted at position i. Therefore, the loop terminates either at the root, i=1, or when the key in the parent of i, which is found at position [image], is smaller than the item to be inserted.

Notice too that a good optimizing compiler will recognize that the subscript calculations involve only division by two. Therefore, the divisions can be replaced by bitwise right shifts which usually run much more quickly.

Since the depth of a complete binary tree with n nodes is [image], the worst case running time for the Enqueue operation is

where [image] is the time required to compare to objects. If

the Enqueue operation is simply [image] in the worst case.


Removing Items from a Binary Heap

The DequeueMin function removes from a priority queue the item having the smallest key. In order to remove the smallest item, it needs first to be located. Therefore, the DequeueMin operation is closely related to FindMin.

The smallest item is always at the root of a min heap. Therefore, the FindMin operation is trivial. Program gif gives the code for the FindMin member function of the BinaryHeap class. Assuming that no exception is thrown, the running time of FindMin is clearly O(1).

program25237
Program: BinaryHeap Class FindMin Member Function Definition

Since the bottom row of a complete tree is filled from left to right as items are added, it follows that the bottom row must be emptied from right to left as items are removed. So, we have a problem: The datum to be removed from the heap by DequeueMin is in the root, but the node to be removed from the heap is in the bottom row.

Figure gif (a) illustrates the problem. The DequeueMin operation removes the key 2 from the heap, but it is the node containing key 6 that must be removed from the tree to make it into a complete tree again. When key 2 is removed from the root, a hole is created in the tree as shown in Figure gif (b).

figure25250
Figure: Removing an Item from a Binary Heap

The trick is to move the hole down in the tree to a point where the left-over key, in this case the key 6, can be reinserted into the tree. To move a hole down in the tree, we consider the children of the empty node and move up the smallest key. Moving up the smallest key ensures that the result will be a min heap.

The process of moving up continues until either the hole has been pushed down to a leaf node, or until the hole has been pushed to a point where the left over key can be inserted into the heap. In the example shown in Figure gif (b)-(c), the hole is pushed from the root node to a leaf node where the key 6 is ultimately placed is shown in Figure gif (d).

Program gif gives the code for the DequeueMin function of the BinaryHeap class. This function implements the deletion algorithm described above. The main loop (lines 9-19) moves the hole in the tree down by moving up the child with the smallest key until either a leaf node is reached or until the hole has been moved down to a point where the last element of the array can be reinserted.

program25641
Program: BinaryHeap Class DequeueMin Member Function Definition

In the worst case, the hole must be pushed from the root to a leaf node. Each iteration of the loop makes at most two object comparisons and moves the hole down one level. Therefore, the running time of the DequeueMin operation is

where [image] is the number of items in the heap and the [image] is the time required to compare two object instances. If

the DequeueMin operation is simply [image] in the worst case.


Leftist Heaps

A leftist heap is a heap-ordered binary tree which has a very special shape called a leftist tree. One of the nice properties of leftist heaps is that is possible to merge two leftist heaps efficiently. As a result, leftist heaps are suited for the implementation of mergeable priority queues.

  • Leftist Trees
  • Implementation
  • Merging Leftist Heaps
  • Putting Items into a Leftist Heap
  • Removing Items from a Leftist Heap


Leftist Trees

A leftist tree is a tree which tends to ``lean'' to the left. The tendency to lean to the left is defined in terms of the shortest path from the root to an external node. In a leftist tree, the shortest path to an external node is always found on the right.

Every node in binary tree has associated with it a quantity called its null path length which is defined as follows:

Definition (Null Path and Null Path Length)

Consider an arbitrary node x in some binary tree T. The null path of node x is the shortest path in T from x to an external node of T.

The null path length of node x is the length of its null path.

Sometimes it is convenient to talk about the null path length of an entire tree rather than of a node:

Definition (Null Path Length of a Tree)

The null path length of an empty tree is zero and the null path length of a non-empty binary tree [image] is the null path length its root R.

When a new node or subtree is attached to a given tree, it is usually attached in place of an external node. Since the null path length of a tree is the length of the shortest path from the root of the tree to an external node, the null path length gives a lower bound on the cost of insertion. For example, the running time for insertion in a binary search tree, Program gif , is at least

where d is the null path length of the tree.

A leftist tree is a tree in which the shortest path to an external node is always on the right. This informal idea is defined more precisely in terms of the null path lengths as follows:

Definition (Leftist Tree) A leftist tree is a binary tree T with the following properties:
  1. Either [image]; or
  2. , where both [image] and [image] are leftist trees which have null path lengths [image] and [image], respectively, such that

Figure gif shows an example of a leftist heap. A leftist heap is simply a heap-ordered leftist tree. The external depth of the node is shown to the right of each node in Figure gif . The figure clearly shows that it is not necessarily the case in a leftist tree that the number of nodes to the left of a given node is greater than the number to the right. However, it is always the case that the null path length on the left is greater than or equal to the null path length on the right for every node in the tree.

figure25684
Figure: A Leftist Heap

The reason for our interest in leftist trees is illustrated by the following theorems:

Theorem Consider a leftist tree T which contains n internal nodes. The path leading from the root of T downwards to the rightmost external node contains at most [image] nodes.

extbfProof Assume that T has null path length d. Then T must contain at least [image] leaves. Otherwise, there would be a shorter path than d from the root of T to an external node.

A binary tree with exactly l leaves has exactly l+1 non-leaf internal nodes. Since T has at least [image] leaves, it must contain at least [image] internal nodes altogether. Therefore, [image].

Since T is a leftist tree, the shortest path to an external node must be the path on the right. Thus, the length of the path to the rightmost external is at most [image].

There is an interesting dichotomy between AVL balanced trees and leftist trees. The shape of an AVL tree satisfies the AVL balance condition which stipulates that the difference in the heights of the left and right subtrees of every node may differ by at most one. The effect of AVL balancing is to ensure that the height of the tree is [image].

On the other hand, leftist trees have an ``imbalance condition'' which requires the null path length of the left subtree to be greater than or equal to that of the right subtree. The effect of the condition is to ensure that the length of the right path in a leftist tree is [image]. Therefore, by devising algorithms for manipulating leftist heaps which only follow the right path of the heap, we can achieve running times which are logarithmic in the number of nodes.

The dichotomy also extends to the structure of the algorithms. For example, an imbalance sometimes results from an insertion in an AVL tree. The imbalance is rectified by doing rotations. Similarly, an insertion into a leftist tree may result in a violation of the ``imbalance condition.'' I.e., the null path length of the right subtree of a node my become greater than that of the left subtree. Fortunately, it is possible to restore the proper condition simply by swapping the left and right subtrees of that node.


Implementation

This section presents a pointer-based implementation of leftist heaps. Program gif gives the declaration of the LeftistHeap class. The LeftistHeap class is derived from both the MergeablePriorityQueue abstract class and the BinaryTree concrete class.

program26092
Program: LeftistHeap Class Definition

  • Member Variables
  • SwapContents Member Function


Member Variables

Since a leftist heap is a heap-ordered binary tree, it inherits from the BinaryTree base class the three member variables: key, left and right. The key is a pointer to the object contained in the given node and the left and right member variables point to the left and right subtrees of the given node, respectively. In addition, the member variable nullPathLength records the null path length of the given node. By recording the null path length in the node, it is possible to check the leftist heap balance condition in constant time.


SwapContents Member Function

In addition to the inherited public interface, the LeftistHeap class definition includes the private member function SwapContents. This routine is used to implement the various operations on leftist heaps. The SwapContents routine takes a reference to another leftist heap node. The effect of the routine is to exchange the contents of ``this'' node with those of the given node. The implementation of SwapContents is trivial and the running time required is obviously O(1).


Merging Leftist Heaps

In order to merge two leftist heaps, say h1 and h2, declared as follows

LeftistHeap h1;
LeftistHeap h2;
we invoke the Merge operation like this:
h1.Merge (h2);
The effect of the Merge routine is to take all the nodes from h2 and to attach them to h1, thus leaving h2 as the empty heap.

In order to achieve a logarithmic running time, it is important for the Merge routine to do all its work on the right sides of h1 and h2. It turns out that the algorithm for merging leftist heaps is actually quite simple.

To begin with, if h1 is the empty heap, then we can simply swap the contents of h1 and h2. Otherwise, let us assume that the root of h2 is larger than the root of h1. Then we can merge the two heaps by recursively merging h2 with the right subheap of h1. After doing so, it may turn out that the right subheap of h1 now has a larger null path length than the left subheap. This we rectify by swapping the left and right subheaps so that the result is again leftist. On the other hand, if h2 initially has the smaller root, we simply exchange the rôles of h1 and h2 and proceed as above.

Figure gif illustrates the merge operation. In this example, we wish to merge the two trees [image] and [image] shown in Figure gif (a). Since [image] has the larger root, it is recursively merged with the right subtree of [image]. The result of that merge replaces the right subtree of [image] as shown in Figure gif (b). Since the null path length of the right subtree is now greater than the left, the subtrees of [image] are swapped giving the leftist heap shown in Figure gif (c).

figure26151
Figure: Merging Leftist Heaps

Program gif gives the code for the Merge member function of the LeftistHeap class. Clearly, the Merge routine only visits nodes on the rightmost paths of the trees being merged. Suppose we are merging two trees, say [image] and [image], with null path lengths [image] and [image], respectively. Then the running time of the Merge routine is

where [image] is time required to compare two keys. If we assume that the time to compare two keys is a constant, then we get [image], where [image] and [image] are the number of internal nodes in trees [image] and [image], respectively.

program26536
Program: LeftistHeap Class Merge Member Function Definition


Putting Items into a Leftist Heap

The Enqueue member function of the LeftistHeap class is used to put items into the heap. Enqueue is easily implemented using the Merge operation. I.e., to enqueue an item in a given heap, we simply create a new heap containing the one item to be enqueued and merge it with the given heap. The algorithm to do this is shown in Program gif .

program26551
Program: LeftistHeap Class Enqueue Member Function Definition

The expression for the running time for the Insert operation follows directly from that of the Merge operation. I.e., the time required for the Insert operation in the worst case is

where d is the null path length of the heap into which the item is inserted. If we assume that two keys can be compared in constant time, the running time for Insert becomes simply [image], where n is the number of nodes in the tree into which the item is inserted.


Removing Items from a Leftist Heap

The FindMin member function locates the item with the smallest key in a given priority queue and the DequeueMin member function removes it from the queue. Since the smallest item in a heap is found at the root, the FindMin operation is easy to implement. Program gif shows how it can be done. Clearly, the running time of the FindMin operation is O(1).

program26570
Program: LeftistHeap Class FindMin Member Function Definitions

Since the smallest item in a heap is at the root, the DequeueMin operation must delete the root node. Since a leftist heap is a binary heap, the root has at most two children. In general when the root is deleted, we are left with two non-empty leftist heaps. Since we already have an efficient way to merge leftist heaps, the solution is to simply merge the two children of the root to obtain a single heap again! Program gif shows how the DequeueMin operation of the LeftistHeap class can be implemented.

program26583
Program: LeftistHeap Class DequeueMin Member Function Definition

The running time of Program gif is determined by the time required to merge the two children of the root (line 13) since the rest of the work in DequeueMin can be done in constant time. Consider the running time to delete the root of a leftist heap T with n internal nodes. The running time to merge the left and right subtrees of T

where [image] and [image] are the null path lengths of the left and right subtrees T, respectively. In the worst case, [image] and [image]. If we assume that [image], the running time for DequeueMin is [image].


A Simple Example-Arithmetic Series Summation

In this section we apply Axioms gif , gif and gif to the analysis of the running time of a program to compute the following simple arithmetic series summation

The algorithm to compute this summation is given in Program gif .

program330
Program: Program to compute

The executable statements in Program gif comprise lines 3-6. Table gif gives the running times of each of these statements.

Table: Computing the running time of Program gif
statement | time | code
3 |  | result = 0
4a |  | i = 1
4b |  | i <= n
4c |  | ++i
5 |  | result += i
6 |  | return result
TOTAL |  |

Note that the for statement on line 4 of Program gif has been split across three lines in Table gif . This is because we analyze the running time of each of the elements of a for statement separately. The first element, the initialization code, is executed once before the first iteration of the loop. The second element, the loop termination test, is executed before each iteration of the loop begins. Altogether, the number of times the termination test is executed is one more than the number of times the loop body is executed. Finally, the third element, the loop counter increment step, is executed once per loop iteration.

Summing the entries in Table gif we get that the running time, T(n), of Program gif is

where [image] and [image].


Binomial Queues

A binomial queue is a priority queue that is implemented not as a single tree but as a collection of heap-ordered trees. A collection of trees is called a forest . Each of the trees in a binomial queue has a very special shape called a binomial tree. Binomial trees are general trees. I.e., the maximum degree of a node is not fixed.

The remarkable characteristic of binomial queues is that the merge operation is similar in structure to binary addition. I.e., the collection of binomial trees that make up the binomial queue is like the set of bits that make up the binary representation of a non-negative integer. Furthermore, the merging of two binomial queues is done by adding the binomial trees that make up that queue in the same way that the bits are combined when adding two binary numbers.

  • Binomial Trees
  • Binomial Queues
  • Implementation
  • Merging Binomial Queues
  • Putting Items into a Binomial Queue
  • Removing an Item from a Binomial Queue


Binomial Trees

A binomial tree is a general tree with a very special shape:

Definition (Binomial Tree) The binomial tree of order [image] with root R is the tree [image] defined as follows
  1. If k=0, [image]. I.e., the binomial tree of order zero consists of a single node, R.
  2. If k>0, [image]. I.e., the binomial tree of order k>0 comprises the root R, and k binomial subtrees, [image], [image], ..., [image].

Figure gif shows the first five binomial trees, [image] - [image]. It follows directly from Definition gif that the root of [image], the binomial tree of order k, has degree k. Since k may arbitrarily large, so too can the degree of the root. Furthermore, the root of a binomial tree has the largest fanout of any of the nodes in that tree.

figure26610
Figure: Binomial Trees [image], [image], ...,

The number of nodes in a binomial tree of order k is a function of k:

Theorem The binomial tree of order k, [image], contains [image] nodes.

extbfProof (By induction). Let [image] be the number of nodes in [image], a binomial tree of order k.

Base Case By definition, [image] consists a single node. Therefore [image].

Inductive Hypothesis Assume that [image] for [image], for some [image]. Consider the binomial tree of order l+1:

Therefore the number of nodes in [image] is given by

Therefore, by induction on l, [image] for all [image].

It follows from Theorem gif that binomial trees only come in sizes that are a power of two. I.e., [image]. Furthermore, for a given power of two, there is exactly one shape of binomial tree.

Theorem The height of [image], the binomial tree of order k, is k.

extbfProof (By induction). Let [image] be the height of [image], a binomial tree of order k.

Base Case By definition, [image] consists a single node. Therefore [image].

Inductive Hypothesis Assume that [image] for [image], for some [image]. Consider the binomial tree of order l+1:

Therefore the height [image] is given by

Therefore, by induction on l, [image] for all [image].

Theorem gif tells us that the height of a binomial tree of order k is k and Theorem gif tells us that the number of nodes is [image]. Therefore, the height of [image] is exactly [image].

Figure gif shows that there are two ways to think about the construction of binomial trees. The first way follows directly from the Definition gif . I.e., binomial [image] consists of a root node to which the k binomial trees [image], [image], ..., [image] are attached as shown in Figure gif (a).

figure26922
Figure: Two Views of Binomial Tree

Alternatively, we can think of [image] as being comprised of two binomial trees of order k-1. For example, Figure gif (b) shows that [image] is made up of two instances of [image]. In general, suppose we have two trees of order k-1, say [image] and [image], where [image]. Then we can construct a binomial tree of order k by combining the trees to get

Why do we call [image] a binomial tree? It is because the number of nodes at a given depth in the tree is determined by the binomial coefficient. And the binomial coefficient derives its name from the binomial theorem. And the binomial theorem tells us how to compute the [image] power of a binomial . And a binomial is an expression which consists of two terms, such as x+y. That is why it is called a binomial tree!

Theorem (Binomial Theorem) The [image] power of the binomial x+y for [image] is given by

where [image] is called the binomial coefficient .

extbfProof The proof of the binomial theorem is left as an exercise for the reader (Exercise gif ). gif

The following theorem gives the expression for the number of nodes at a given depth in a binomial tree:

Theorem The number of nodes at level l in [image], the binomial tree of order k, where [image], is given by the binomial coefficient [image].

extbfProof (By induction). Let [image] be the number of nodes at level l in [image], a binomial tree of order k.

Base Case Since [image] contains a single node, there is only one level in the tree, l=0, and exactly one node at that level. Therefore, [image].

Inductive Hypothesis Assume that [image] for [image], for some [image]. The binomial tree of order h+1 is composed of two binomial trees of height h, one attached under the root of the other. Hence, the number of nodes at level l in [image] is equal to the number of nodes at level l in [image] plus the number of nodes at level l-1 in [image]:

Therefore by induction on h, [image].


Binomial Queues

If binomial trees only come in sizes that are powers of two, how do we implement a container which holds an arbitrary number number of items n using binomial trees? The answer is related to the binary representation of the number n. Every non-negative integer n can be expressed in binary form as

where [image] is the [image] binary digit or bit in the representation of n. For example, n=27 is expressed as the binary number [image] because 27=16+8+2+1.

To make a container which holds exactly n items we use a collection of binomial trees. A collection of trees is called a forest . The forest contains binomial tree [image] if the [image] bit in the binary representation of n is a one. I.e., the forest [image] which contains exactly n items is given by

where [image] is determined from Equation gif . For example, the forest which contains 27 items is

The analogy between [image] and the binary representation of n carries over to the merge operation. Suppose we have two forests, say [image] and [image]. Since [image] contains n items and [image] contains m items, the combination of the two contains n+m items. Therefore, the resulting forest is [image].

For example, consider n=27 and m=10. In this case, we need to merge [image] with [image]. Recall that two binomial trees of order k can be combined to obtain a binomial tree of order k+1. E.g., [image]. But this is just like adding binary digits! In binary notation, the sum 27+10 is calculated like this:

 |  | 1 | 1 | 0 | 1 | 1
+ |  |  | 1 | 0 | 1 | 0
 | 1 | 0 | 0 | 1 | 0 | 1

The merging of [image] and [image] is done in the same way:

+

Therefore, the result is [image].


Implementation

  • Heap-Ordered Binomial Trees
  • Binomial Queues
  • Member Variables
  • AddTree and RemoveTree
  • FindMinTree and FindMin Member Functions


Heap-Ordered Binomial Trees

Since binomial trees are simply general trees with a special shape, we can make use of the GeneralTree class presented in Section gif to implement the BinomialTree class. As shown in Program gif , the BinomialTree class is derived from the GeneralTree class from which it inherits almost all its functionality. In addition to the constructor, only two more member functions are declared--SwapContents and Add.

No new member variables a declared in the BinomialTree class. Remember that the implementation of the GeneralTree class uses a linked list to contain the pointers to the subtrees, since the degree of a node in a general tree may be arbitrarily large. Also, the GeneralTree class already keeps track of the degree of a node in its member variable degree. Since the degree of the root node of a binomial tree of order k is k, it is not necessary to keep track of the order explicitly. The degree variable serves this purpose nicely.

program27221
Program: BinomialTree Class Definition

The purpose of the SwapContents member function is evident from its name--it simply exchanges the contents of two nodes of a binomial tree. It is relatively easy to implement an algorithm to swap the contents of a node of a binomial tree. It is not difficult to ensure that the running time of SwapContents is O(1), regardless of the degrees of the nodes whose contents are swapped.

The Add member function is used to combine two binomial trees of the same order, say k, into a single binomial tree of order k+1. Each of the two trees to be combined is heap-ordered. Since the smallest key is at the root of a heap-ordered tree, we know that the root of the result must be the smaller root of the two trees which are to be combined. Therefore, to combine the two trees, we simply attach the tree with the larger root under the root of the tree with the smaller root. For example, Figure gif illustrates how two heap-ordered binomial trees of order two are combined into a single heap-ordered tree of order three.

figure27238
Figure: Adding Binomial Trees

The implementation of the Add function is given in Program gif . The Add function takes a reference to a BinomialTree and attaches the specified tree to this node. This is only permissible when both trees have the same order.

program27466
Program: BinomialTree Class Add Member Function Definition

In order to ensure that the resulting binomial tree is heap ordered, the roots of the trees are compared. If necessary, the contents of the nodes are exchanged using SwapContents (lines 5-6) before the subtree is attached (line 7). Clearly the running time of the Add member function is

I.e., exactly one comparison and a constant amount of additional work is needed to combine two binomial trees.


Binomial Queues

A binomial queue is a mergeable priority queue implemented as a forest of binomial trees. In this section we present a pointer-based implementation of the forest. I.e., the forest is represented using a linked list of pointers to binomial trees.

Program gif defines the BinomialQueue class. The BinomialQueue is derived from the MergeablePriorityQueue abstract base class.

program27482
Program: BinomialQueue Class Definition


Member Variables

The BinomialQueue class definition contains the single member variable list, which is declared as a linked list of pointers to BinomialTree instances. The binomial trees contained in the linked list are stored in increasing order. I.e., the binomial tree at the head of the list will have the smallest order, and the binomial tree at the tail will have the largest order.


AddTree and RemoveTree

The private member functions AddTree and RemoveTree of the BinomialQueue class facilitate the implementation of the various priority queue operations. These functions are defined in Program gif . The AddTree function takes a reference to a BinomialTree and appends a pointer to that tree to list. AddTree also adjusts the count in order to keep track of the number of items in the priority queue. It is assumed that the order of the tree which is added is larger than all the others in the list and, therefore, that it belongs at the end of the list. The running time of AddTree is clearly O(1).

program27519
Program: BinomialQueue Class AddTree and RemoveTree Member Function Definitions

The RemoveTree function takes a reference to a binomial tree and removes it from the list. It is assumed that the specified tree is actually in the list. RemoveTree also adjust the count as required. The running time of RemoveTree depends on the position of the tree in the list. A binomial queue which contains exactly n items altogether has at most [image] binomial trees. Therefore, the running time of RemoveTree is [image] in the worst case.


FindMinTree and FindMin Member Functions

A binomial queue that contains n items consists of at most [image] binomial trees. Each of these binomial trees is heap ordered. In particular, the smallest key in each binomial tree is at the root of that tree. So, we know that the smallest key in the queue is found at the root of one of the binomial trees, but we do not know which tree it is.

The private member function FindMinTree is used to determine which of the binomial trees in the queue has the smallest root. As shown in Program gif , the FindMinTree simply traverses the entire linked list to find the tree with the smallest key at its root. Since there are at most [image] binomial trees, the worst-case running time of FindMinTree is

program27543
Program: BinomialQueue Class FindMinTree and FindMin Member Function Definitions

Program gif also defines the public FindMin function which returns the smallest key in the priority queue. The FindMin function uses FindMinTree to locate the tree with the smallest key at its root and returns a reference to that key. Clearly, the asymptotic running time of FindMin is the same as that of FindMinTree.


Merging Binomial Queues

Merging two binomial queues is like doing binary addition. For example, consider the addition of [image] and [image]:

+

The usual algorithm for addition begins with the least-significant ``bit.'' Since [image] contains a [image] tree and [image] does not, the result is simply the [image] tree from [image].

In the next step, we add the [image] from [image] and the [image] from [image]. Combining the two [image] s we get a [image] which we carry to the next column. Since there are no [image] s left, the result does not contain any. The addition continues in a similar manner until all the columns have been added up.

Program gif gives an implementation of this addition algorithm. The Merge member function of the BinomialQueue class takes a reference to a BinomialQueue and adds its subtrees to this binomial queue.

program27585
Program: BinomialQueue Class Merge Member Function Definition

Each iteration of the main loop of the algorithm (lines 12-24) computes the [image] ``bit'' of the result--the [image] bit is a binomial tree of order i. At most three terms need to be considered: the carry from the preceding iteration and two [image] s, one from each of the queues that are being merged.

Two functions, Sum and Carry, compute the result required in each iteration. Program gif defines both Sum and Carry. Notice that the Sum function simply selects and returns one of its arguments. Therefore, the running time for Sum is clearly O(1).

program27603
Program: BinomialQueue Class Sum and Carry Member Function Definitions

In the worst case, the Carry function calls the Add function to combine two BinomialTrees into one. Therefore, the worst-case running time for Carry is

Suppose the Merge routine of Program gif is used to combine a binomial queue with n items with another that contains m items. Since the resulting priority queue contains n+m items, there are at most [image] binomial trees in the result. Thus, the running time for the Merge operation is


Array Subscripting Operations

We now address the question of accessing the elements of an array of data. In general, the elements of a one-dimensional array are stored in consecutive memory locations. Therefore, given the address of the first element of the array, a simple addition suffices to determine the address of an arbitrary element of the array:

Axiom The time required for the address calculation implied by an array subscripting operation, e.g., a[i], is a constant, [image]. This time does not include the time to compute the subscript expression, nor does it include the time to access (i.e., fetch or store) the array element.

By applying Axiom gif , we can determine that the running time for the statement

y = a [i];
is [image]. Three operand fetches are required: the first to fetch a, the base address of the array;, the second to fetch i, the index into the array; and, the third to fetch array element a[i].


Putting Items into a Binomial Queue

With the Merge routine at our disposal, the Enqueue operation is easy to implement. To enqueue an item in a given binomial queue, we create another binomial queue that contains just the one item to be enqueued and merge that queue with the original one.

Program gif shows how easily this can be done. Creating the empty queue (line 3) takes a constant amount of time. Creating the binomial tree [image] with the one object at its root (line 4) can also be done in constant time. Finally, the time required to merge the two queues is

where n is the number of items originally in the queue.

program27626
Program: BinomialQueue Class Enqueue Member Function Definition


Removing an Item from a Binomial Queue

A binomial queue is a forest of heap-ordered binomial trees. Therefore, to dequeue the smallest item from the queue, we must withdraw the root of one of the binomial trees. But what do we do with the rest of the tree once its root has been removed?

The solution lies in realizing that the collection of subtrees of the root of a binomial tree is a forest! For example, consider the binomial tree of order k,

Taken all together, its subtrees form the binomial queue [image]:

Therefore, to delete the smallest item from a binomial queue, we first identify the binomial tree with the smallest root and remove that tree from the queue. Then, we consider all the subtrees of the root of that tree as a binomial queue and merge that queue back into the original one. Program gif shows how this can be coded.

program27641
Program: BinomialQueue Class DequeueMin Member Function Definition

The DequeueMin function begins by calling FindMinTree to find the tree with the smallest root and then removing that tree using RemoveTree (lines 6-7). The time required to find the appropriate tree and to remove it is

where n is the number of items in the queue.

A new binomial queue is created on line 9. All the children of the root of the minimum tree are detached from the tree and added to the new binomial queue (lines 10-15). In the worst case, the minimum tree is the one with the highest order. i.e., [image], and the root of that tree has [image] children. Therefore, the running time of the loop on lines 10-15 is [image].

The new queue is then merged with the original one (line 16). Since the resulting queue contains n-1 keys, the running time for the Merge operation in this case is

The remaining operations (lines 19-21) are just housekeeping which can be done in constant time.


Applications

  • Discrete Event Simulation
  • Implementation


Discrete Event Simulation

One of the most important applications of priority queues is in discrete event simulation. Simulation is a tool which is used to study the behavior of complex systems. The first step in simulation is modeling. We construct a mathematical model of the system we wish to study. Then we write a computer program to evaluate the model. In a sense the behavior of the computer program mimics the system we are studying.

The systems studied using discrete event simulation have the following characteristics: The system has a state which evolves or changes with time. Changes in state occur at distinct points in simulation time. A state change moves the system from one state to another instantaneously. State changes are called events.

For example, suppose we wish to study the service received by customers in a bank. Suppose a single teller is serving customers. If the teller is not busy when a customer arrives at the bank, the that customer is immediately served. On the other hand, if the teller is busy when another customer arrives, that customer joins a queue and waits to be served.

We can model this system as a discrete event process as shown in Figure gif . The state of the system is characterized by the state of the server (the teller), which is either busy or idle, and by the number of customers in the queue. The events which cause state changes are the arrival of a customer and the departure of a customer.

figure27665
Figure: A Simple Queueing System

If the server is idle when a customer arrives, the server immediately begins to serve the customer and therefore changes state its to busy. If the server is busy when a customer arrives, that customer joins the queue.

When the server finishes serving the customer, that customer departs. If the queue is not empty, the server immediately commences serving the next customer. Otherwise, the server becomes idle.

How do we keep track of which event to simulate next? Each event (arrival or departure) occurs at a discrete point in simulation time . In order to ensure that the simulation program is correct, it must compute the events in order. This is called the causality constraint--events cannot change the past.

In our model, when the server begins to serve a customer we can compute the departure time of that customer. So, when a customer arrives at the server we schedule an event in the future which corresponds to the departure of that customer. In order to ensure that events are processed in order, we keep them in a priority queue in which the time of the event is its priority. Since we always process the pending event with the smallest time next and since an event can schedule new events only in the future, the causality constraint will not be violated.


Implementation

This section presents the simulation of a system comprised of a single queue and server as shown in Figure gif . Program gif declares the class Event which represents the events. There are two parts to an event, a Type (either arrival or departure), and a Time (a double).

program27885
Program: Event Class Definition

Since events will be put into a priority queue, the Event class is derived from the Association class which is defined in Section gif . An association is an ordered pair comprised of a key and a value. In the case of the Event class, the key is the time of the event and the value is the type of the event. Therefore, the events in a priority queue are prioritized by their times.

Program gif defines the function which implements the discrete event simulation. This function takes two arguments. The first, eventList, is a reference to a priority queue. This priority queue is used to hold the events during the course of the simulation. The second parameter, timeLimit, specifies the total amount of time to be simulated.

program27898
Program: Application of Priority Queues--Discrete Event Simulation

The state of the system being simulated is represented by the two variables serverBusy and numberInQueue. The first is a Boolean value which indicates whether the server is busy. The second keeps track of the number of customers in the queue.

In addition to the state variables, there are two instances of the class ExponentialRV. The class ExponentialRV is a random number generator defined in Section gif . This class provides a member function called Sample which is used to sample the random number generator. I.e., every time Sample is called, a different (random) result is returned. The random values are exponentially distributed around a mean value which is specified in the constructor. For example, in this case both serviceTime and interArrivalTime produce random distributions with the mean value of 100 (lines 5-6).

It is assumed that the eventList priority queue is initially empty. The simulation begins by enqueuing a customer arrival at time zero (line 8). The while loop (lines 9-46) constitutes the main simulation loop. This loop continues as long as the eventList is not empty, i.e., as long as there is an event to be simulated

Each iteration of the simulation loop begins by dequeuing the next event in the event list (lines 11-12). If the time of that event exceeds timeLimit, the event is discarded, the eventList is purged, and the simulation is terminated. Otherwise, the simulation proceeds.

The simulation of an event depends on the type of that event. The switch statement (line 20) invokes the appropriate code for the given event. If the event is a customer arrival and the server is not busy, serverBusy is set to true and the serviceTime random number generator is sampled to determine the amount of time required to service the customer. A customer departure is scheduled at the appropriate time in the future (lines 25-27). On the other hand, if the server is already busy when the customer arrives, we add one to the numberInQueue variable (line 30).

Another customer arrival is scheduled after every customer arrival. The interArrivalTime random number generator is sampled, and the arrival is scheduled at the appropriate time in the future (lines 31-32).

If the event is a customer departure and the queue is empty, the server becomes idle (lines 35-36). When a customer departs and there are still customers in the queue, the next customer in the queue is served. Therefore, numberInQueue is decreased by one and the serviceTime random number generator is sampled to determine the amount of time required to service the next customer. A customer departure is scheduled at the appropriate time in the future (lines 39-41).

Clearly the execution of the Simulation routine given in Program gif mimics the modeled system. Of course, the program given produces no output. For it to be of any practical value, the simulation program should be instrumented to allow the user to study its behavior. For example, the user may be interested in knowing statistics such as the average queue length and the average waiting time that a customer waits for service. And such instrumentation can be easily incorporated into the given framework.


Exercises

  1. For each of the following key sequences determine the binary heap obtained when the keys are inserted one-by-one in the order given into an initially empty heap:
    1. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
    2. 3, 1, 4, 1, 5, 9, 2, 6, 5, 4.
    3. 2, 7, 1, 8, 2, 8, 1, 8, 2, 8.
  2. For each of the binary heaps obtained in Exercise gif determine the heap obtained after three consecutive DequeueMin operations.
  3. Repeat Exercises gif and gif for a leftist heap.
  4. Show the result obtained by inserting the keys [image] one-by-one in the order given into an initially empty binomial queue.
  5. A full binary tree is a tree in which each node is either a leaf or its is a full node (see Exercise gif ). Consider a complete binary tree with n nodes.
    1. For what values of n is a complete binary tree a full binary tree.
    2. For what values of n is a complete binary a perfect binary tree.
  6. Prove by induction Theorem gif .
  7. Devise an algorithm to determine whether a given binary tree is a heap. What is the running time of your algorithm?
  8. Devise an algorithm to find the largest item in a binary min heap. Hint: First, show that the largest item must be in one of the leaves. What is the running time of your algorithm?
  9. Suppose we are given an arbitrary array of n keys to be inserted into a binary heap all at once. Devise an O(n) algorithm to do this. Hint: See Section gif .
  10. Devise an algorithm to determine whether a given binary tree is a leftist tree. What is the running time of your algorithm?
  11. Prove that a complete binary tree is a leftist tree.
  12. Suppose we are given an arbitrary array of n keys to be inserted into a leftist heap all at once. Devise an O(n) algorithm to do this. Hint: See Exercises gif and gif .
  13. Consider a complete binary tree with its nodes numbered as shown in Figure gif . Let K be the number of a node in the tree. The the binary representation of K is

    where [image].

    1. Show that path from the root to a given node K passes passes through the following nodes:

    2. Consider a complete binary tree with n nodes. The nodes on the path from the root to the [image] are special. Show that every non-special node is the root of a perfect tree.
  14. The Enqueue algorithm for the BinaryHeap class does [image] object comparisons in the worst case. In effect, this algorithm does a linear search from a leaf to the root to find the point at which to insert a new key. Devise an algorithm that a binary search instead. Show that the number of comparisons required becomes [image]. Hint: See Exercise gif .
  15. Prove Theorem gif .
  16. Do Exercise gif .


Projects

  1. Design and implement a sorting algorithm using one of the priority queue implementations described in this chapter.
  2. Complete the BinaryHeap class declared in Program gif by providing suitable definitions for the following member functions: CompareTo, IsFull, Accept and NewIterator. Write a test program and test your implementation.
  3. Complete the LeftistHeap class declared in Program gif by providing suitable definitions for the following member functions: LeftistHeap (two constructors), Left, Right and SwapContents. You must also have a complete implementation of the base class BinaryTree. (See Project gif ). Write a test program and test your implementation.
  4. Complete the implementation of the BinomialTree class declared in Program gif by providing suitable definitions for the following member functions: BinomialTree (constructor), Count, Subtree and SwapContents. You must also have a complete implementation of the base class GeneralTree. (See Project gif ). Write a test program and test your implementation.
  5. Complete the implementation of the BinomialQueue class declared in Program gif by providing suitable definitions for the following member functions: BinomialQueue (constructor), BinomialQueue (destructor), Purge, CompareTo, Accept and NewIterator. You must also have a complete implementation of the BinomialTree class. (See Project gif ). Write a test program and test your implementation.
  6. The binary heap described in this chapter uses an array as the underlying foundational data structure. Alternatively we may base an implementation on the BinaryTree class described in Chapter gif . Implement a concrete priority queue class using multiple inheritance to inherit the functionality from the BinaryTree class (Program gif ) and the interface from the PriorityQueue class (Program gif ).
  7. Implement a concrete priority queue class using the binary search tree class from Chapter gif . Specifically, use multiple inheritance to inherit the functionality from the BST class (Program gif ) and the interface from the PriorityQueue class (Program gif ). You will require a complete implementation of the base class BST. (See Project gif ). Write a test program and test your implementation.
  8. Devise and implement an algorithm to multiply two polynomials:

    Generate the terms of the result in order by putting intermediate product terms into a priority queue. I.e., use the priority queue to group terms with the same exponent. Hint: See also Project gif .


Sets, Multisets and Partitions

In mathematics a set is a collection of elements, especially a collection having some feature or features in common. The set may have a finite number of elements, e.g., the set of prime numbers less than 100; or it may have an infinite number of elements, e.g., the set of right triangles. The elements of a set may be anything at all--from simple integers to arbitrarily complex objects. However, all the elements of a set are distinct--a set may contain only one instance of a given element.

For example, [image], [image], [image] and [image] are all sets the elements of which are drawn from [image]. The set of all possible elements, U, is called the universal set . Note also that the elements comprising a given set are not ordered. Thus, [image] and [image] are the same set.

There are many possible operations on sets. In this chapter we consider the most common operations for combining sets--union, intersection, difference:

union
The union (or conjunction ) of sets S and T, written [image], is the set comprised of all the elements of S together with all the element of T. Since a set cannot contain duplicates, if the same item is an element of both S and T, only one instance of that item appears in [image]. If [image] and [image], then [image].
intersection
The intersection (or disjunction ) of sets S and T is written [image]. The elements of [image] are those items which are elements of both S and T. If [image] and [image], then [image].
difference
The difference (or subtraction ) of sets S and T, written S-T, contains those elements of S which are not also elements of T. I.e., the result S-T is obtained by taking the set S and removing from it those elements which are also found in T. If [image] and [image], then [image].

Figure gif illustrates the basic set operations using a Venn diagram . A Venn diagram represents the membership of sets by regions of the plane. In Figure gif the two sets S and T divide the plane into the four regions labeled I-IV. The following table illustrates the basic set operations by enumerating the regions that comprise each set.

figure28117
Figure: Venn Diagram Illustrating the Basic Set Operations

set | region(s) of Figure gif
U | I, II, III, IV
S | I, II
S' | III, IV
T | II, III
 | I, II, III
 | II
S-T | I
T-S | III

  • Basics
  • Array and Bit-Vector Sets
  • Multisets
  • Partitions
  • Applications
  • Exercises
  • Projects


Basics

In this chapter we consider sets the elements of which are integers. By using integers as the universe rather than arbitrary objects, certain optimizations are possible. For example, we can use a bit-vector of length N to represent a set whose universe is [image]. Of course, using integers as the universe does not preclude the use of more complex objects, provided there is a one-to-one mapping between those objects and the elements of the universal set.

A crucial requirement of any set representation scheme is that it supports the common set operations including union , intersection and set difference . We also need to compare sets and, specifically, to determine whether a given set is a subset of another.

  • Implementing Sets


Implementing Sets

figure28183
Figure: Object Class Hierarchy

As discussed above, this chapter addresses the implementation of sets of integers. A set is a collection of elements. Naturally, we want to insert and withdraw objects from the collection and to test whether a given object is a member of the collection. Therefore, we consider sets as being derived from the SearchableContainer class defined in Chapter gif .

In general, a searchable container can hold arbitrary objects. However, in this chapter we will assume that the elements of a set are integers. Furthermore, all the searchable container implementations which we have seen so far have been based on two assumptions. These are that the container owns the objects it contains and that indirect storage is used, i.e., a pointer to the contained object is actually held by the container. Since we deal with integers and not with arbitrary objects, the set implementations in this chapter invalidate both these assumptions.

Program gif defines the abstract class Set. The Set class is derived from SearchableContainer which is defined in Section gif . The definition does not declare any new member functions--the interface inherited from the base class is sufficient. In addition, a new type called Set::Element is defined.

program28199
Program: Set Class Definition

The items contained in a set are unsigned integers. However, the member functions of the base class interface such as Insert, IsMember, and Withdraw, expect their arguments to be derived from the class Object. Therefore, the Wrapper template (defined in Section gif ) is used to define the type Set::Element as the encapsulation of an unsigned int. We assume that the only Object instances which are passed to a set are instances of the class Set::Element.

The default constructor for the Set class is also given in Program gif . It takes a single argument, [image], which specifies that the universal set shall be [image].


Another Example-Horner's Rule

In this section we apply Axioms gif , gif , gif and gif to the analysis of the running time of a program which evaluates the value of a polynomial. I.e., given the n+1 coefficients [image], and a value x, we wish to compute the following summation

The usual way to evaluate such polynomials is to use Horner's rule , which is an algorithm to compute the summation without requiring the computation of arbitrary powers of x. The algorithm to compute this summation is given in Program gif . Table gif gives the running times of each of the executable statements in Program gif .

program394
Program: Program to compute [image] using Horner's rule

Table: Computing the running time of Program gif
statement | time
3 |
4a |
4b |
4c |
5 |
6 |
TOTAL |

Summing the entries in Table gif we get that the running time, T(n), of Program gif is

where [image] and [image].


Array and Bit-Vector Sets

In this section we consider finite sets over a finite universe. Specifically, the universe we consider is [image], the set of integers in the range from zero to N-1, for some fixed and relatively small value of N.

Let [image] be the universe. Every set which we wish to represent is a subset of U. The set of all subsets of U is called the power set of U and is written [image]. Thus, the sets which we wish to represent are the elements of [image]. The number of elements in the set U, written |U|, is N. Similarly, [image]. This observation should be obvious: For each element of the universal set U there are only two possibilities: Either it is, or it is not, a member of the given set.

This suggests a relatively straightforward representation of the elements of [image] --an array of Boolean values, one for each element of the universal set. By using array subscripts in U, we can represent the set implicitly. I.e., i is a member of the set if the [image] array element is true.

Program gif declares the class SetAsArray. This class uses an array of length [image] to represent the elements of [image] where [image]. A SetAsArray is a Set. Therefore, it supports the basic operations of searchable containers including Insert, IsMember, and Withdraw.

program28238
Program: SetAsArray Class Definition

In addition, Program gif overloads the operators +, *, -, ==, and <=. The first three operators correspond to set union, set intersection, and set difference (respectively). The last two are used to compare two sets and to determine whether one set is a subset of another.

  • Basic Operations
  • Union, Intersection and Difference
  • Comparing Sets
  • Bit-Vector Sets


Basic Operations

Program gif defines the constructor for the SetAsArray class as well as the three basic operations--Insert, IsMember, and Withdraw. The constructor takes a single argument [image], which defines the universe and, consequently, the size of the array of Boolean values. The constructor creates the empty set by initializing all the element of the Boolean array to false. Clearly, the running time of the constructor is O(N).

program28266
Program: SetAsArray Class Constructor, Insert, Withdraw and IsMember Member Function Definitions

The Insert function is used to put an item into the set. The function takes a reference to an Object instance which is assumed to be a Set::Element. A dynamic cast is used to ensure that the object to be inserted is of the correct type and to extract the integer value from that object. Then the corresponding element of array is set to true to indicate that the specified object has been added to the set. Note that the set does not keep track of the actual object instance that was inserted. Therefore, the set cannot own contained objects. Instead, objects that are inserted are represented implicitly by array indices. The running time of the Insert operation is O(1).

The IsMember function is used to test whether a given item is an element of the set. The semantics are somewhat subtle. Since a set does not actually keep track of the specific object instances that are inserted, the membership test is based on the value of the argument. Again, a dynamic cast is used to ensure that the argument is of the correct type and to extract the integer value from that object. The function simply returns the value of the appropriate element of the array. The running time of the IsMember operation is O(1).

The Withdraw member function is used to take an item out of a set. The withdrawal operation is the opposite of insertion. Instead of setting the appropriate array element to true, it is set to false. The running time of the Withdraw is identical to that of Insert, viz., is O(1).


Union, Intersection and Difference

Program gif overloads the three operators, operator+, operator* and operator-, to provide the union, intersection, and difference operations for SetAsArray instances. Operators +, * and - correspond to [image], [image] and -, respectively.

program28308
Program: SetAsArray Class Union, Intersection and Difference Operator Definitions

The set union operator takes two references to SetAsArray instances, s and t, and computes a result of type SetAsArray. The implementation given requires that the s and t sets be compatible. Two sets are deemed to be compatible if they have the same universe. The result also has the same universe. Consequently, the Boolean array in all three sets has the same length, N. The set union routine creates a result array of the required size and then computes the elements of the array as required. The [image] element of the result is true if either the [image] element of s or the [image] element of t is true. Thus, set union is implemented using the Boolean or operator, ||.

The set intersection operator is almost identical to set union, except that the elements of the result are computed using the Boolean and operator. The set difference operator is also very similar. In this case, an item is an element of the result only if it is a member of s and not a member of t.

Because all three routines are almost identical, their running times are essentially the same. I.e., the running time of the set union, intersection, and difference operations is O(N), where [image].


Comparing Sets

There is a special family of operators for comparing sets. Consider two sets, say S and T. We say that S is a subset of T, written [image], if every element of S is also an element of T. If there is at least one element of T that is not also an element of S, we say that S is a proper subset of T, written [image]. We can also reverse the order in which the expressions are written to get [image] or [image], which indicates that T is a (proper) superset of S.

The set comparison operators follow the rule that if [image] and [image] then [image], which is analogous to a similar property of numbers: . However, set comparison is unlike numeric comparison in that there exist sets S and T for which neither [image] nor [image]! E.g., clearly this is the case for [image] and [image]. Mathematically, the relation [image] is called a partial order because there exist some pairs of sets for which neither [image] nor [image] holds; whereas the relation [image] (among integers, say) is a total order.

Program gif overloads the operators == and <= for SetAsArray operands. The former tests its operands for equality and the latter determines whether the relation [image] holds between its operands. Both operators return a Boolean result. The worst-case running time of each of these operations is clearly O(N).

program28356
Program: SetAsArray Class Comparison Operator Definitions

A complete repertoire of comparison operators would also include definitions for <, >, >= and !=. These operations follow directly from the implementation shown in Program gif (Exercise gif ).


Bit-Vector Sets

In the typical C++ implementation, a bool occupies between one and four bytes. However, since there are only the two values true and false, a single bit is sufficient to hold a Boolean value. Therefore, we can realize a significant reduction in the memory space required to represent a set if we use an array of bits. Furthermore, by using bitwise operations to implement the basic set operations such as union and intersection, we can achieve a commensurate reduction in execution time. Unfortunately, these improvements are not free--the operations Insert, IsMember and Withdraw, all slow down by a constant factor.

Since C++ does not directly support arrays of bits, we must simulate an array of bits using an array of machine words. Program gif illustrates how this can be done. The class SetAsBitVector represents the elements of a the set using the bits in an array of unsigned integers (i.e., type Word). The enumerated constant wordBits is defined as the number of bits in a single Word.

program28382
Program: SetAsBitVector Class Definition

  • Basic Operations
  • Union, Intersection and Difference


Basic Operations

Program gif defines the constructor for the SetAsBitVector class as well as the three basic operations--Insert, IsMember, and Withdraw. The constructor takes a single argument [image], which specifies the universe and, consequently, the number of bits needed in the bit array. The constructor creates a vector of Words of length [image], where [image] is the number of bits in a word, and sets the elements of the vector to zero. The running time of the constructor is [image].

program28400
Program: SetAsBitVector Class Constructor, Insert, Withdraw and IsMember Member Function Definitions

To insert an item into the set, we need to change the appropriate bit in the array of bits to one. The [image] bit of the bit array is bit [image] of word [image]. Thus, the Insert function is implemented using a bitwise or operation to change the [image] bit to one as shown in Program gif . Even though it is slightly more complicated than the corresponding operation for the SetAsArray class, the running time for this operation is still O(1). Since [image] is a power of two, it is possible to replace the division and modulo operations, / and %, with shifts and masks like this:

vector [item >> shift] |= 1 << (item & mask);
for a suitable definition of the constants shift and mask. Depending on the compiler and machine architecture, doing so may improve the performance of the Insert operation by a constant factor. Of course, its asymptotic performance is still O(1).

To withdraw an item from the set, we need to clear the appropriate bit in the array of bits and to test if an item is a member of the set, we test the corresponding bit. The IsMember and Withdraw routines in Program gif show how this can be done. Like Insert, both these routines have constant worst-case running times.


Union, Intersection and Difference

The implementations of the union, intersection, and difference operators (+, *, and -, respectively) for operands of type SetAsBitVector are shown in Program gif . The code is quite similar to that for the SetAsArray class given in Program gif .

program28441
Program: SetAsBitVector Class Union, Intersection and Difference Operator Definitions

Instead of using the Boolean operators &&, || and !, we have used the bitwise operators &, | and . By using the bitwise operators, [image] bits of the result are computed in each iteration of the loop. Therefore, the number of iterations required is [image] instead of N. The worst-case running time of each of these operations is [image].

Notice that the asymptotic performance of these SetAsBitVector class operations is the same as the asymptotic performance of the SetAsArray class operations. I.e., both of them are O(N). Nevertheless, the SetAsBitVector class operations are faster. In fact, the bit-vector approach is asymptotically faster than the the array approach by the factor w.


Multisets

A multiset is a set in which an item may appear more than once. I.e., whereas duplicates are not permitted in a regular set, they are permitted in a multiset. Multisets are also known simply as bags .

Sets and multisets are in other respects quite similar: Both support operations to insert and withdraw items; both provide a means to test the membership of a given item; and both support the basic set operations of union, intersection, and difference. As a result, the Set abstract class and the Multiset abstract class share a common interface as shown in Program gif .

program28468
Program: Multiset Class Definition

  • Array Implementation
  • Linked List Implementation


Array Implementation

A regular set may contain either zero or one instance of a particular item. As shown in the preceding section if the number of possible items is not excessive, we may use an array of Boolean variables to keep track of the number of instances of a particular item in a regular set. The natural extension of this idea for a multiset is to keep a separate count of the number of instances of each item in the multiset.

Program gif declares the class MultisetAsArray. The multiset is implemented using an array of [image] counters. Each counter is an unsigned int in this case.

program28480
Program: MultisetAsArray Class Definition

  • Basic Operations
  • Union, Intersection and Difference


Basic Operations

Program gif defines the constructor for the MultisetAsArray class as well as the three basic operations--Insert, IsMember, and Withdraw. The constructor takes a single argument, [image], and initializes an array of length N counters all to zero. The running time of the constructor is O(N).

program28496
Program: MultisetAsArray Class Constructor, Insert, Withdraw and IsMember Member Function Definitions

To insert an item, we simply increase the appropriate counter; to delete an item, we decrease the counter; and to test whether an item is in the set, we test whether the corresponding counter is greater than zero. In all cases the operation can be done in constant time.


Goals

The primary goal of this book is to promote object-oriented design using C++ and to illustrate the use of the emerging object-oriented design patterns. Experienced object-oriented programmers find that certain ways of doing things work best and that these ways occur over and over again. The book shows how these patterns are used to create good software designs. In particular, the following design patterns are used throughout the text: singleton, container, iterator, adapter and visitor.

Virtually all of the data structures are presented in the context of a single, unified, polymorphic class hierarchy. This framework clearly shows the relationships between data structures and it illustrates how polymorphism and inheritance can be used effectively. In addition, algorithmic abstraction is used extensively when presenting classes of algorithms. By using algorithmic abstraction, it is possible to describe a generic algorithm without having to worry about the details of a particular concrete realization of that algorithm.

A secondary goal of the book is to present mathematical tools just in time. Analysis techniques and proofs are presented as needed and in the proper context. In the past when the topics in this book were taught at the graduate level, an author could rely on students having the needed background in mathematics. However, because the book is targeted for second- and third-year students, it is necessary to fill in the background as needed. To the extent possible without compromising correctness, the presentation fosters intuitive understanding of the concepts rather than mathematical rigor.


Analyzing Recursive Functions

In this section we analyze the performance of a recursive algorithm which computes the factorial of a number. Recall that the factorial of a non-negative integer n, written n!, is defined as

However, we can also define factorial recursively as follows

It is this latter definition which leads to the algorithm given in Program gif to compute the factorial of n. Table gif gives the running times of each of the executable statements in Program gif .

program433
Program: Recursive program to compute n!

Table: Computing the running time of Program gif
 | time
statement | n=0 | n>0
3 |  |
4 |  | --
6 | -- |

Notice that we had to analyze the running time of the two possible outcomes of the conditional test on line 3 separately. Clearly, the running time of the program depends on the result of this test.

Furthermore, the function Factorial calls itself recursively on line 6. Therefore, in order to write down the running time of line 6, we need to know the running time, [image], of Factorial. But this is precisely what we are trying to determine in the first place! We escape from this catch-22 by assuming that we already know what is the function [image], and that we can make use of that function to determine the running time of line 6.

By summing the columns in Table gif we get that the running time of Program gif is

where [image] and [image]. This kind of equation is called a recurrence relation because the function is defined in terms of itself recursively.

  • Solving Recurrence Relations-Repeated Substitution


Union, Intersection and Difference

Because multisets permit duplicates but sets do not, the definitions of union, intersection and difference are slightly modified for multisets. The union of multisets S and T, written [image], is the multiset comprised of all the elements of S together with all the element of T. Since a multiset may contain duplicates, it does not matter if the same element appears in S and T.

The subtle difference between union of sets and union of multisets gives rise to an interesting and useful property. If S and T are regular sets,

On the other hand, if S and T are multisets,

The intersection of sets S and T is written [image]. The elements of [image] are those items which are elements of both S and T. If a given element appears more than once in S or T (or both), the intersection contains m copies of that element, where m is the smaller of the number of times the element appears in S or T. E.g., if [image] and [image], the intersection is [image].

The difference of sets S and T, written S-T, contains those elements of S which are not also elements of T. I.e., the result S-T is obtained by taking the set S and removing from it those elements which are also found in T.

Program gif gives the implementations of the union, intersection, and difference operators (+, *, and -, respectively) for operands of type MultisetAsArray. This code is quite similar to that of the SetAsArray class (Program gif ) and the SetAsBitVector class (Program gif ). The worst-case running time of each of these operations is O(N).

program28532
Program: MultisetAsArray Class Union, Intersection and Difference Operator Definitions

Instead of using the Boolean operators &&, || and !, we have used + (integer addition), Min and - (integer subtraction). The following table summarizes the operators used in the various set and multiset implementations.

 | class
operation | SetAsArray | SetAsBitVector | MultisetAsArray
union | || | | | +
intersection | && | & | Min
difference | && and ! | & and | <= and -


Linked List Implementation

The array implementation of multisets is really only practical if the number of items in the universe, N=|U|, is not too large. If N is large, then it is impractical, or at least extremely inefficient, to use an array of N counters to represent the multiset. This is especially so if the number of elements in the multisets is significantly less than N.

If we use a linked list of elements to represent a multiset S, the space required is proportional to the size of the multiset, |S|. When the size of the multiset is significantly less than the size of the universe, [image], it is more efficient in terms of both time and space to use a linked list.

Program gif gives the declaration of the MultisetAsLinkedList class. The MultisetAsLinkedList class is a concrete class derived from the abstract base class Multiset. In this case a linked list of unsigned ints is used to record the contents of the multiset.

How should the elements of the multiset be stored in the list? Perhaps the simplest way is to store the elements in the list in no particular order. Doing so makes the Insert operation efficient--it can be done in constant time. Furthermore, the IsMember and Withdraw operations both take O(n) time, where n is the number of items in the multiset, regardless of the order of the items in the linked list.

Consider now the union, intersection, and difference of two multisets, say S and T. If the linked list is unordered, the worst case running time for the union operation is O(m+n), where m=|S| and n=|T|. Unfortunately, intersection and difference are both O(mn).

If, on the other hand, we use an ordered linked list, union, intersection, and difference can all be done in O(m+n) time. The trade-off is that the insertion becomes an O(n) operation rather than a O(1). The MultisetAsLinkedList implementation presented in this section records the elements of the multiset in an ordered linked list.

program28584
Program: MultisetAsLinkedList Class Definition

  • Union
  • Intersection


Union

The union operation for MultisetAsLinkedList class requires the merging of two ordered, linked lists as shown in Program gif . We have assumed that the smallest element contained in a multiset is found at the head of the linked list and the largest is at the tail.

program28596
Program: MultisetAsLinkedList Class Union Operator Definition

The union operator takes two multisets and computes a third multiset, the result, as follows. The main loop of the program (lines 9-21) traverses the linked lists of the two operands, in each iteration appending the smallest remaining element the result. Once one of the lists has been exhausted, the remaining elements in the other list are simply appended to the result (lines 22-25). The total running time for the union operation, operator+, is O(m+n), where [image] and [image] and s and t are the two operand multisets.


Intersection

The implementation of the intersection operator for the MultisetAsLinkedList class is similar to that of union. However, instead of merging of two ordered, linked lists to construct a third, we compare the elements of two lists and append an item to the third only when it appears in both of the input lists. The intersection operator, operator*, is shown in Program gif .

program28613
Program: MultisetAsLinkedList Class Intersection Operator Definition

The main loop of the program traverses the linked lists of both input operands at once using two pointers (lines 9-18). If the next element in each list is the same, that element is appended to the result and both pointers are advanced. Otherwise, only one of the pointers is advanced--the one pointing to the smaller element.

The number of iterations of the main loop actually done depends on the contents of the respective linked lists. The best case occurs when both lists are identical. In this case, the number of iterations is m, where [image]. In the worst case case, the number of iterations done is m+n. Therefore, the running time of the intersection operation, operator*, is O(m+n).


Partitions

Consider the finite universal set [image]. A partition of U is a finite set of sets [image] with the following properties:

  1. The sets [image], [image], ..., [image] are pairwise disjoint. I.e.,

  2. The sets [image], [image], ..., [image] span the universe U. I.e.,

For example, consider the universe [image]. There are exactly five partitions of U:

In general, given a universe U of size n>0, i.e., |U|=n, there are [image] partitions of U, where [image] is the so-called Stirling number of the second kind which denotes the number of ways to partition a set of n elements into m nonempty disjoint subsets. gif

Applications which use partitions typically start with an initial partition and refine that partition either by joining or by splitting elements of the partition according to some application-specific criterion. The result of such a computation is the partition obtained when no more elements can be split or joined.

In this chapter we shall consider only applications that begin with the initial partition of U in which each item in U is in a separate element of the partition. Thus, the initial partition consists of |U| sets, each of size one (like [image] above). Furthermore, we restrict the applications in that we only allow elements of a partition to be joined--we do not allow elements to split.

The two operations to be performed on partitions are:

Find
Given an item in the universe, say [image], find the element of the partition that contains i. I.e., find [image] such that [image].
Join
Given two distinct elements of a partition P, say [image] and [image] such that [image], create a new partition P' by removing the two elements [image] and [image] from P and replacing them with a single element [image].

For example, consider the partition [image]. The result of the operation [image] is the set [image] because 3 is a member of [image]. Furthermore, when we join sets [image] and [image], we get the partition [image].

  • Representing Partitions
  • Implementing a Partition using a Forest
  • Collapsing Find
  • Union by Size
  • Union by Height or Rank


Representing Partitions

Program gif declares the abstract class Partition. Since a partition is a set of sets, it makes sense to use a class derived from the abstract base class Set defined in Section gif . The two member functions, Find and Join, correspond to the partition operations described above.

program28678
Program: Partition Class Definition

The elements of a partition are also sets. Consequently, the objects contained in a Partition are also derived from the Set class. The Find member function of the Partition class expects as its argument a reference to a Set::Element and returns a reference to the Set which contains the specified element.

The Join member function takes two arguments, both of them references to Set instances. The two arguments are expected to be distinct elements of the partition. The effect of the Join operation is to remove the specified sets from the partition and replace them with a Set which represents the union of the two.


Implementing a Partition using a Forest

A partition is a set of sets. Consequently, there are two related issues to consider when developing an approach for representing partitions:

  1. How are the individual elements or parts of the partition represented?
  2. How are the elements of a partition combined into the whole?
This section presents an approach in which each element of a partition is a tree. Therefore, the whole partition is a forest .

For example, Figure gif shows how the partition

can be represented using a forest. Notice that each element of the universal set [image] appears in exactly one node of exactly one tree.

figure28709
Figure: Representing a Partition as a Forest

The trees in Figure gif have some very interesting characteristics. The first characteristic concerns the shapes of the trees: The nodes of the trees have arbitrary degrees. The second characteristic concerns the positions of the keys: there are no constraints on the positions of the keys in a tree. The final characteristic has to do with the way the tree is represented: Instead of pointers to its children, each node of a tree contains only one pointer--a pointer to its parent!

Since there is no particular order to the nodes in the trees, it is necessary to keep track of the position of each node explicitly. Figure gif shows how this can be done using an array of pointers. (This figure shows the same partition as in Figure gif ). The array contains a pointer for each element of the universal set U. Specifically, the [image] array element contains a pointer to the node that contains item i. Having found the desired node, we can follow the chain of parent pointers to find the root of the corresponding tree.

figure28936
Figure: Finding the Elements of a Partition

  • Implementation
  • Constructors and Destructor
  • Find and Join Member Functions


Implementation

Program gif declares two classes--PartitionTree and PartitionAsForest. The former is used to represent the individual elements or parts of a partition and the latter encapsulates all of the parts that make up a given partition.

program29196
Program: PartitionTree and PartitionAsForest Class Definitions

The PartitionTree class is derived from the abstract base classes Set and Tree. Since we are representing the parts of a partition using trees, it makes sense that we derive them from the Tree class. On the other hand, since a partition is a set of sets, we must also derive the parts of a partition from the Set class. In particular, this is necessary because of the way that the Find and Join member functions are defined.

The PartitionTree class has three member variables--parent, item and rank. Each instance of this class represents one node of a tree. The parent member variable points to the parent of a given node and the item member variable records the element of the universal set that the given node represents. The remaining variable, rank, is optional. While it is not required in order to provide the basic functionality, as shown below, the rank variable can be used in the implementation of the Join operation to improve the performance of subsequent Find operations.

The PartitionAsForest class represents a complete partition. Since a partition is a set of sets, it is a container. And since it supports a find operation, the class PartitionAsForest is derived from the abstract base class SearchableContainer. The PartitionAsForest class contains a single member variable, array, which is an array of pointers to PartitionTree instances. The [image] element of the array always points to the tree node that contains element i of the universe.


Constructors and Destructor

Program gif gives the code for the PartitionTree constructor. The constructor creates a tree comprised of a single node. It takes an argument which specifies the element of the universal set that the node is to represent. The parent field is set to zero to indicate that the node has no parent. Consequently, the node is a root node. Finally, the rank field is initialized to zero. The running time of the constructor is O(1).

program29241
Program: PartitionTree and PartitionAsForest Class Constructor and Destructor Definitions

Program gif also shows the constructor and destructor for the PartitionAsForest class. The constructor takes a single argument N which specifies that the universe shall be [image]. It creates an initial partition of the universe consisting of N parts. Each part contains one element of the universal set and, therefore, comprises a one-node tree. The PartitionAsForest is a container and the PartitionTree instances that the constructor creates are the contained objects.

The destructor is responsible for deleting the objects in the container. I.e., it deletes all the PartitionTree instances that were created in the constructor. Both the constructor and destructor run in O(N) time, where N is the number of elements in the universe.


Find and Join Member Functions

Two elements of the universe are in the same part of the partition if and only if they share the same root node. Since every tree has a unique root, it makes sense to use the root node as the ``handle'' for that tree. Therefore, the find operation takes an element of the universal set and returns the root node of the tree that contains that element. And because of way in which the trees are represented, we can follow the chain of parent pointers to find the root node.

Program gif gives the code for the Find member function of the PartitionAsForest class. The Find function takes as its lone argument a reference to an Object instance and returns a reference to a Set. The argument is expected to be actually a Set::Element that specifies the item of the universe that is the object of the search.

program29269
Program: PartitionAsForest Class Find Member Function Definition

The Find operation begins at the node array[item] and follows the chain of parent pointers to find the root node of the tree that contains the specified item. The result of the function is a reference to the root node.

The running time of the Find operation is O(d) where d is the depth in the tree of the node from which the search begins. If we don't do anything special to prevent it, the worst case running time is O(N), where N is the size of the universe. The best performance is achieved when every non-root node points to the root node. In this case, the running time is O(1).

Another advantage of having a pointer to the parent in each node is that the join operation can be implemented easily and efficiently. For example, suppose we wish to join the two sets [image] and [image] shown in Figure gif . While there are many possible representations for [image], it turns out that there are two simple alternatives which can be obtained in constant time. These are shown in Figure gif . In the first alternative, the root of [image] is made a child of the root of [image]. This can be done in constant time simply by making the parent pointer of the root of [image] point to the root of [image]. The second alternative is essentially the same as the first except that the rôles of [image] and [image] are exchanged.

figure29285
Figure: Alternatives for Joining Elements of a Partition

Program gif gives the simplest possible implementation for the Join operation. The Join member function of the PartitionAsForest class takes two arguments--both of the references to Sets. Both arguments are required to be references to distinct PartitionTree instances which are contained in the given partition. Furthermore, both of them are required to be root nodes. Therefore, the sets that the arguments represent are disjoint. The member function CheckArguments makes sure that the arguments satisfy these conditions.

The Join operation is trivial and executes in constant time: It simply makes one node the parent of the other. In this case, we have arbitrarily chosen that the node specified by the first argument shall always become the parent.

program29613
Program: PartitionAsForest Class Simple Join Member Function Definition


Solving Recurrence Relations-Repeated Substitution

In this section we present a technique for solving a recurrence relation such as Equation gif called repeated substitution . The basic idea is this: Given that [image], then we may also write [image], provided n>1. Since T(n-1) appears in the right-hand side of the former equation, we can substitute for it the entire right-hand side of the latter. By repeating this process we get

The next step takes a little intuition: We must try to discern the pattern which is emerging. In this case it is obvious:

where [image]. Of course, if we have doubts about our intuition, we can always check our result by induction:

Base Case Clearly the formula is correct for k=1, since [image].

Inductive Hypothesis Assume that [image] for [image]. By this assumption

Note also that using the original recurrence relation we can write

for [image]. Substituting Equation gif in the right-hand side of Equation gif gives

Therefore, by induction on l, our formula is correct for all [image].

So, we have shown that [image], for [image]. Now, if n was known, we would repeat the process of substitution until we got T(0) on the right hand side. The fact that n is unknown should not deter us--we get T(0) on the right hand side when n-k=0. I.e., k=n. Letting k=n we get

where [image] and [image].


Collapsing Find

Unfortunately, using the Join algorithm given in Program gif can result in particularly bad trees. E.g., Figure gif shows the worst possible tree that can be obtained. Such a tree is bad because its height is O(N). In such a tree both the worst case and the average case running time for the Find operation is O(N).

figure29630
Figure: A Degenerate Tree

There is an interesting trick we can play that can improve matters significantly. Recall that the find operation starts from a given node and locates the root of the tree containing that node. If, having found the root, we replace the parent pointer of the given node with a pointer to the root, the next time we do a Find it will be more efficient.

In fact, we can go one step further and replace the parent pointer of every node along the search path to the root. This is called a collapsing find operation. Doing so does not change the asymptotic complexity of the Find operation. However, a subsequent Find operation which begins at any point along the search path to the root will run in constant time!

Program gif gives the code for a collapsing version of the Find operation. The Find function first determines the root node as before. Then, a second pass is made up the chain from the initial node to the root, during which the parent pointer of each node is made to point at the root. Clearly, this version of Find is slower than the one given in Program gif because it makes two passes up the chain rather than one. However, the running of this version of Find is still O(d), where d is the depth of the node from which the search begins.

program29839
Program: PartitionAsForest Class Collapsing Find Member Function Definition

Figure gif illustrates the effect of a collapsing find operation. After the find, all the nodes along the search path are attached directly to the root. I.e., they have had their depths decreased to one. As a side-effect, any node which is in the subtree of a node along the search path may have its depth decreased by the collapsing find operation. The depth of a node is never increased by the find operation. Eventually, if we do enough collapsing find operations, it is possible to obtain a tree of height one in which all the non-root nodes point directly at the root.

figure29849
Figure: Example of Collapsing Find


Union by Size

While using collapsing find does mitigate the negative effects of poor trees, a better approach is to avoid creating bad trees in the first place. As shown in Figure gif , when we join to trees we have a choice--which node should we choose to be the root of the new tree. A simple, but effective choice is to attach the smaller tree under the root of the larger one. In this case, the smaller tree is the one which has fewer nodes. This is the so-called union-by-size join algorithm. Program gif shows how this can be done.

program29999
Program: PartitionAsForest Class Union-by-Size Join Member Function Definition

The implementation uses the count field of the Container class, from which PartitionTree is derived, to keep track of the number of items contained in the tree. (Since each node contains one item from the universal set, the number of items contained in a tree is equal to the number of nodes in that tree). The algorithm simply selects the tree with the largest number of nodes to become the root of the result and attaches the root of the smaller tree under that of the larger one. Clearly, the running time of the union-by-size version of Join is O(1).

The following theorem shows that when using the union-by-size join operation, the heights of the resulting trees grow logarithmically.

Theorem Consider an initial partition P of the universe [image] comprised of N sets of size 1. Let S be an element of the partition obtained from P after some sequence of union-by-size join operations, such that |S|=n for some [image]. Let T be the tree representing the set S. The height of tree T satisfies the inequality

extbfProof (By induction).

Base Case Since a tree comprised of a single node has height zero, the theorem clearly holds for n=1.

Inductive Hypothesis Suppose the theorem holds for trees containing n nodes for [image] for some [image]. Consider a union-by-size join operation that produces a tree containing k+1 nodes. Such a tree is obtained by joining a tree [image] having [image] nodes with another tree [image] that has [image] nodes, such that l+m=k+1.

Without loss of generality, suppose [image]. As a result, l is less than or equal to m. Therefore, the union-by-size algorithm will attach [image] under the root of [image]. Let [image] and [image] be the heights of [image] and [image] respectively. The height of the resulting tree is [image].

According to the inductive hypothesis, the height of [image] is given by

Similarly, the quantity [image] is bounded by

Therefore, the height of the tree containing k+1 nodes is no greater than [image]. By induction on k, the theorem holds for all values of [image].

Note that Theorem gif and its proof does not require that we use the collapsing find algorithm of Section gif . I.e., the height of a tree containing n nodes is guaranteed to be [image] when the simple find is used. Of course, there is nothing precluding the use of the collapsing find in conjunction with the union-by-size join routine. And doing so only makes things better.


Union by Height or Rank

The union-by-size join algorithm described above controls the heights of the trees indirectly by basing the join algorithm on the sizes of the trees. If we explicitly keep track of the height of a node in the node itself, we can accomplish the same thing.

Program gif gives an implementation of the Join routine that always attaches the shorter tree under the root of the taller one. This routine assumes that the rank field is used to keep track of the height of a node. (The reason for calling it rank rather than height will become evident shortly).

program30032
Program: PartitionAsForest Class Union-by-Rank Join Member Function Definition

The only time that the height of node increases is when joining two trees that have the same height. In this case, the height of the root increases by exactly one. If the two trees being joined have different heights, attaching the shorter tree under the root of the taller one has no effect on the height of the root.

Unfortunately, there is a slight complication if we combine union-by-height with the collapsing find. Since the collapsing find works by moving nodes closer to the root, it affects potentially the height of any node moved. It is not at all clear how to recompute efficiently the heights that have changed. The solution is not to do it at all!

If we don't recompute the heights during the collapsing find operations, then the heights will no longer be exact. Nevertheless, the quantities remain useful estimates of the heights of nodes. We call the estimated height of a node its rank and the join algorithm which uses rank instead of height is called union by rank .

Fortunately, Theorem gif applies equally well when when union-by-rank is used. I.e., the height of tree which contains n nodes is [image]. Thus, the worst-case running time for the Find operation grows logarithmically with n. And as before, collapsing find only makes things better.


Applications

One of the most important applications of partitions involves the processing of equivalence relations. Equivalence relations arise in many interesting contexts. For example, two nodes in an electric circuit are electrically equivalent if there is a conducting path (a wire) connecting the two nodes. In effect, the wires establish an electrical equivalence relation over the nodes of a circuit.

A similar relation arises among the user-defined data types in a C++ program. Consider the following C++ code fragment:

class A;
typedef A B;
typedef A C;
typedef B D;
The four data types A, B, C and D are equivalent in the sense that values of one type can be assigned directly to variables of another (without requiring a type conversion). In effect, the typedef declarations establish a type equivalence relation over the user-defined data types in a C++ program.

Definition (Equivalence Relation) An equivalence relation over a universal set U is a relation [image] with the following properties:
  1. The relation [image] is reflexive . I.e., for every [image], [image].
  2. The relation [image] is symmetric . I.e., for every pair [image] and [image], if [image] then [image].
  3. The relation [image] is transitive . I.e., for every triple [image], [image] and [image], if [image] and [image] then [image].

An important characteristic of an equivalence relation is that it partitions the elements of the universal set U into a set of equivalence classes . I.e., U is partitioned into [image], such that for every pair [image] and [image], [image] if and only if x and y are in the same element of the partition. I.e.,

For example, consider the universe [image]. and the equivalence relation [image] defined over U defines as follows:

This relation results in the following partition of U:

The list of equivalences in Equation gif contains many redundancies. Since we know that the relation [image] is reflexive, symmetric and transitive, it is possible to infer the complete relation from the following list

The problem of finding the set of equivalence classes from a list of equivalence pairs is easily solved using a partition. Program gif shows how it can be done using the PartitionAsForest class defined in Section gif .

program30077
Program: Application of Disjoint Sets--Finding Equivalence Classes

The algorithm first gets a positive integer n from the input and creates a partition, p, of the universe [image] (lines 3-5). As explained in Section gif , the initial partition comprises n disjoint sets of size one. I.e., each element of the universal set is in a separate element of the partition.

Each iteration of the main loop processes one equivalence pair (lines 9-18). An equivalence pair consists of two numbers, i and j, such that [image] and [image]. The find operation is used to determine the sets s and t in partition p that contain elements i and j, respectively (lines 11-12).

If s and t are not the same set, then the disjoint sets are united using the join operation (lines 13-14). Otherwise, i and j are already in the same set and the equivalence pair is redundant (lines 15-17). After all the pairs have been processed, the final partition is printed (line 19).


Exercises

  1. For each of the following implementations derive an expression for the total memory space required to represent a set which contains of n elements drawn from the universe [image].
    1. SetAsArray (Program gif ),
    2. SetAsBitVector (Program gif ),
    3. MultisetAsArray (Program gif ), and
    4. MultisetAsLinkedList (Program gif ).
  2. In addition to = and [image], a complete repertoire of set operators includes [image], [image], [image] and [image]. For each of the set implementations listed in Exercise gif show how to implement the remaining operators.
  3. The symmetric difference of two sets S and T, written [image] is given by

    For each of the set implementations listed in Exercise gif devise an algorithm to compute symmetric difference. What is the running time of your algorithm?

  4. The complement of a set S over universe U, written S' is given by

    Devise an algorithm to compute the complement of a set represented as a bit vector. What is the running time of your algorithm?

  5. Devise an algorithm to sort a list of integers using a multiset. What is the running time of your algorithm? Hint: See Section gif .
  6. Consider a multiset implemented using linked lists. When the multiset contains duplicate items, each of those items occupies a separate list element. An alternative is to use a linked list of ordered pairs of the form [image] where i an the element of the universal set U and [image] is a non-negative integer that counts the number of instances of the element i in the multiset.

    Derive an expression for the total memory space required to represent a multiset which contains of n instances of m distinct element drawn from the universe [image].

  7. Consider a multiset implemented as described in Exercise gif . Devise algorithms for set union, intersection and difference. What are the running times of your algorithms?
  8. Consider the initial partition [image]. For each of the methods of computing the union listed below show the result of the following sequence join operations: , [image], [image], [image], [image], [image], [image], [image], [image].
    1. simple union,
    2. union by size,
    3. union by height, and
    4. union by rank.
  9. For each final partition obtained in Exercise gif , show the result of performing a collapsing find operation for item 9.
  10. Consider the initial partition P of the universe [image] comprised of N sets[19].
    1. Show that N-1 join operations can be performed before the number of elements in the partition is reduced to one.
    2. Show that if n join operations are done ( [image]), the size of the largest element of the partition is at most n+1.
    3. A singleton is an element of a partition that contains only one element of the universal set. Show that when n join operations are done ( [image]), at least [image] singletons are left.
    4. Show that if less that [image] join operations are done, at least one singleton is left.


Projects

  1. Complete the SetAsArray class declared in Program gif by providing suitable definitions for the following member functions: SetAsArray (destructor), Purge, IsEmpty, IsFull, Count, Accept, and NewIterator. Write a test program and test your implementation.
  2. Complete the SetAsBitVector class declared in Program gif by providing suitable definitions for the following member functions: SetAsBitVector (destructor), Purge, IsEmpty, IsFull, Count, Accept, and NewIterator. Write a test program and test your implementation.
  3. Rewrite the Insert, Withdraw and IsMember member functions of the SetAsBitVector implementation so that they use bitwise shift and mask operations rather than division and modulo operations. Compare the running times of the modified routines with the original ones and explain your observations.
  4. Complete the MultisetAsArray class declared in Program gif by providing suitable definitions for the following member functions: MultisetAsArray (destructor), Purge, Count, Accept, and NewIterator. Write a test program and test your implementation.
  5. Complete the MultisetAsLinkedList class declared in Program gif by providing suitable definitions for the following member functions: SetAsArray (destructor), Purge, IsEmpty, IsFull, Count, CompareTo, Accept, and NewIterator. Write a test program and test your implementation.
  6. Design and implement a multiset class in which the contents of the set are represented by a linked list of ordered pairs of the form [image], where i an the element of the universal set U and [image] is a non-negative integer that counts the number of instances of the element i in the multiset. (See Exercises gif and gif ).
  7. Write a program to compute the number of ways in which a set of n elements can be partitioned. I.e., compute [image] where

    Hint: See Section gif .


Dynamic Storage Allocation: The Other Kind of Heap

In the preceding chapters when analyzing an algorithm involving dynamically allocated storage, we assume that the time taken to acquire or to release storage is bounded by a constant. Specifically, Axiom gif in Chapter gif states that the running time of operator new is a constant, [image], and that the running time of operator delete is also a constant, [image].

But is this really so? To answer this question, we consider in this chapter the dynamic management of a pool of memory. In particular, we consider several different implementations for the operators new and delete and we show that the assumptions of constant running times are not always valid.

  • Basics
  • Singly Linked Free Storage
  • Doubly Linked Free Storage
  • Buddy System for Storage Management
  • Applications
  • Exercises
  • Projects


Basics

A memory pool is a region of contiguous memory locations that is subdivided into non-overlapping areas which are then allocated to the user. For example, Figure gif shows the memory map of a 128KB pool of memory. The gray bars denote the blocks of memory that are currently in use; the blank areas are unreserved and available for use.

figure30316
Figure: Memory Map of a Storage Pool

A memory pool supports the following two operations:

Acquire
The acquire operation locates in the pool a region of contiguous, unused memory locations of a specified size and returns a pointer to that region. The region is marked reserved, which indicates that the memory locations contained therein are in use.

Should there be insufficient memory to satisfy the request, the acquire operation fails--the typical implementation throws an exception.

Release
The release operation returns a region of reserved memory locations to the pool. After a region is returned to the pool, it is no longer reserved and can be allocated in a subsequent acquire operation.

It is an error to attempt to release a region which is not reserved. The behavior of the pool under these circumstances is typically undefined.

Program gif gives the definition of the StoragePool abstract class. This class serves as the base class from which concrete memory pool implementations are derived. In addition to the virtual destructor, two pure virtual member functions, Acquire and Release, are defined.

program30333
Program: StoragePool Class Definition

The Acquire function takes a single argument of type sizet and its return value is a pointer to void. The purpose of this function is to acquire a region of memory locations, the size of which is specified by the argument (in bytes). A pointer to the region is returned. We shall assume that if for any reason the request cannot be satisfied, an exception of type badalloc is thrown.

The Release function takes as its lone argument a pointer to void. This pointer is assumed to be a pointer previously returned by the Acquire function. I.e., it is a pointer to a region of reserved memory locations. The purpose of this function is to return the storage to the memory pool.

  • C++ Magic
  • The Heap


C++ Magic

Program gif defines the abstract interface, StoragePool. I.e., given an instance p of a concrete class derived from StoragePool, say SomePool, we can call the member functions like this:

SomePool p;
void* ptr = p.Acquire (100);
// ...
p.Release (ptr);
This code sequence first acquires and subsequently releases a 100-byte area of memory.

What is the relationship between the StoragePool class and the operators new and delete? In a C++ program, the new operator is typically used in a statement of the form

T* tptr = new T;
where T is the name of a type. The new operator creates a new instance of type T and returns a pointer to that instance. This operation involves three distinct steps:
  1. Sufficient storage to hold an instance of type T (i.e., [image] bytes) is acquired; and then,
  2. the constructor for T is called to initialize the the object instance; and then,
  3. a pointer to the object is returned.
The C++ compiler accomplishes the first step by calling the function operator new, the prototype of which is:
void* operator new (size_t);
I.e., the function operator new takes an argument that specifies the amount of storage to acquire (in bytes), and returns a pointer to that storage.

Similarly, the statement delete tptr releases the object to which tptr points and returns it to the pool of available storage. This operation involves two distinct steps:

  1. The destructor for object in question is called to finalize the object instance; and then,
  2. the memory locations once occupied by the object are released.
The C++ compiler accomplishes the second step by calling the function operator delete, the prototype of which is:
void operator delete (void*);

The programmer can take over the management of dynamic storage by overloading (or redefining) the functions operator new and operator delete. For example, given an instance p of a concrete class derived from StoragePool, say SomePool, we can overload the functions like this:

SomePool p;

void* operator new (size_t bytes)
    { return p.Acquire (bytes); }

void operator delete (void* vptr)
    { p.Release (vptr); }
In this case, the storage pool is represented by a statically allocated global variable p. All dynamic storage is acquired from and released back to the storage pool p. gif

  • Working with Multiple Storage Pools


Working with Multiple Storage Pools

C++ provides the means for the programmer to control the placement of dynamically allocated storage. The placement syntax provides additional arguments to the new operator like this:

T* tptr = new (argument list...) T;
The extra arguments in the argument list are passed to a suitably overloaded version of operator new.

For example, suppose that we provide an operator new function with the following definition:

void* operator new (size_t bytes, StoragePool& p)
    { return p.Acquire (bytes); }
I.e., we have declared a version of operator new that takes two arguments. The first specifies the number of bytes to be allocated, and the second is a reference to an instance of a storage pool from which the storage is to be acquired. We can then write the following statement:
SomePool p, q;
T* tptr1 = new (p) T;
T* tptr2 = new (q) T;
Two distinct storage pools are declared--p and q. The first instance of T is allocated in the pool p, whereas the second instance is allocated in the pool q.

Unfortunately, but not without good cause, it is not possible to overload operator delete in an analogous fashion. There is no explicit way to return storage to a specific pool using operator delete. I.e., the only way to release storage is by using operator delete like this:

delete tptr1;
delete tptr2;
What we would like to happen is for the first delete to invoke the Release member function of the pool p, and for the second delete to invoke the Release function of the pool q.

One way to achieve this is to keep track explicitly of the pool from which the memory was acquired in each block of dynamically allocated storage. We can accomplish this by attaching to each block a tag which contains a pointer to a pool to which it belongs. Program gif shows how this can be done.

program30419
Program: Overloading operator new and operator delete

The Tag struct encapsulates the tag which is attached to each block of dynamically allocated storage. In this implementation, the tag appears in memory immediately before the memory address returned to the caller. This means that when N bytes of storage are required, [image] bytes are actually allocated. The tag occupies the first [image] bytes. However, it is a pointer to the remaining N bytes that is returned to the user.

Two versions of operator new are defined in Program gif . The first version makes use of the placement syntax described above to allow the user to specify the storage pool from which to allocate storage. This version saves a pointer to the pool from which the storage is allocated in the adjacent tag (lines 6-12). Notice that the function returns [image] which is the address of the memory location which immediately follows the tag.

The second version of operator new is the version that gets invoked when the user does not use the placement syntax to specify the memory pool from which to acquire the storage. In this case, since no storage pool is specified, the standard C library routine malloc is called. And since no pool has been specified, the adjacent tag field is set to zero (lines 14-20).

Since every block of dynamically allocated storage will have been tagged, the delete operator can determine the pool to which the storage is to be returned from the tag field. As shown in Program gif , operator delete obtains a pointer to a StoragePool from the tag and calls the Release member function of that pool if the pointer is nonzero. If the pointer is zero, the malloc routine was used to acquire the storage. Therefore, the C library routine free is called to release the storage (lines 22-29).

Given that we have defined the operations as shown in Program gif and that we have at our disposal a concrete storage pool class, say SomePool, we can safely write the following program fragment:

SomePool p, q;
T* tptr0 = new T;
T* tptr1 = new (p) T;
T* tptr2 = new (q) T;
// ...
delete tptr0;
delete tptr1;
delete tptr2;
Each of the three instances of class T is allocated in a different memory pool. Nevertheless, each object is properly returned to the pool from which it came by the delete operation!


Yet Another Example-Finding the Largest Element of an Array

In this section we consider the problem of finding the largest element of an array. I.e., given an array of n non-negative integers, [image], we wish to find

The straightforward way of solving this problem is to perform a linear search of the array. The linear search algorithm is given in Program gif and the running times for the various statements are given in Table gif .

program493
Program: Linear search to find

Table: Computing the running time of Program gif
statement | time
3 |
4a |
4b |
4c |
5 |
6 |
7 |

With the exception of line 6, the running times follow simply from Axioms gif , gif and gif . In particular, note that the body of the loop is executed n-1 times. This means that the conditional test on line 5 is executed n-1 times. However, the number of times line 6 is executed depends on the data in the array and not just n.

If we consider that in each iteration of the loop body, the variable result contains the largest array element seen so far, then line 6 will be executed in the [image] iteration of the loop only if [image] satisfies the following

Thus, the running time of Program gif , [image], is a function not only of the number of elements in the array, n, but also of the actual array values, [image]. Summing the entries in Table gif we get

where

While this result may be correct, it is not terribly useful. In order to determine the running time of the program we need to know the number of elements in the array, n, and we need to know the values of the elements in the array, [image]. Even if we know these data, it turns out that in order to compute the running time of the algorithm, [image], we actually have to solve the original problem!


The Heap

A storage pool in which regions of memory are dynamically allocated is often called a heap . For example, in C++ the space for a variable is allocated essentially in one of three possible places: Global variables are allocated in the space of initialized static variables ; the local variables of a procedure are allocated in the procedure's activation record , which is typically found in the processor stack ; and dynamically allocated variables are allocated in the heap. In this context the term heap is taken to mean the storage pool for dynamically allocated variables.

In Chapter gif we consider heaps and heap-ordered trees in the context of priority queue implementations. Unfortunately, the only thing that the heaps of Chapter gif and the heap considered here have in common is the name. While it may be possible to use a heap (in the sense of Definition gif ) to manage a dynamic storage pool, typical implementations do not. In this context the technical meaning of the term heap is closer to its dictionary definition--``a pile of many things.''[9]


Singly Linked Free Storage

The objective in the implementation of a storage pool is to make the running times for Acquire and Release operations as small as possible. Ideally, both operations run in constant time. In this section, we present a storage pool implementation that uses a singly-linked list to keep track of the unused areas of memory. The consequence of using a this approach is that the running times are not ideal.

There are several requirements that the implementation of a storage pool must satisfy: It must keep track somehow of the blocks of memory that have been allocated as well as the areas of memory that remain unallocated.

For example, in order to implement the Acquire operation, we must have the means to locate an unused area of memory of sufficient size in order to satisfy the request. The approach taken in this section is to use a singly-linked list to keep track of the free areas in the pool.

In addition to keeping track of the free areas, it is necessary to keep track of the size of each block that is allocated. This is necessary because the Release operation takes only a pointer to the block of memory to be released. I.e., the size of the block is not provided as an argument to the Release function.

Where should we keep track of this extra information? It turns out that the usual approach is to keep the necessary information in the storage pool itself. An area that has not been allocated to a user is available for use by the pool itself. Specifically, the nodes of the linked list of free areas themselves occupy the free areas.

We implement the storage pool as an array of Blocks. The structure of a Block is shown in Figure gif . A sequence of consecutive, contiguous blocks in the array constitutes an area. Only the first block in each area is used to keep track of the entire area.

figure30480
Figure: SinglyLinkedPool::Block Structure Layout

An area which has been allocated is said to be reserved . The first word of the first block in the area is used to keep track of the length of the area (in blocks). The remaining memory locations in the area are given up to the user.

An area which has not been allocated is said to be free . The first word of the first block in the area is used to keep track of the length of the area (in blocks). All of the free areas are linked together in a singly-linked list, known as the free list . The second word of the first block in the area contains a pointer to the next free area in the free list. For reasons explained below, we keep the free list sorted by the address of areas contained therein.

  • Implementation


Implementation

Program gif gives the declaration of the concrete storage pool class SinglyLinkedPool. This class is derived from the abstract base class StoragePool. The public interface of the class comprises a constructor, the destructor, and the two member functions Acquire and Release.

program30561
Program: SinglyLinkedPool Class Definition

The two nested struct definitions, Header and Block, correspond to the structure layout shown in Figure gif . Specifically, the Header structure contains information which appears in the first block of an area, regardless of whether it is reserved or free. The Block structure contains a union. In C++, the elements of a union occupy the same space. I.e., if the block is reserved, the userPart of the block has been allocated to the user. On the other hand, if the block is free, the next field is used to point to the next element of the free list.

Notice how the size of the Block structure is controlled. In this case, the size is set to 16 bytes. Recall that an area is a set of contiguous blocks. Therefore, the size of an area is a multiple of 16 bytes. The selection of a block size of 16 represents a trade-off. If the block size is too large, then space is wasted when a user requests an amount of storage that that is not a multiple of 16. On the other hand, if the block size is too small, then it is possible for the memory pool to become excessively fragmented. E.g., it is possible for the free list to contain many small areas. The choice of 16 is somewhat arbitrary. However, values between 8 and 16 are typical. In this implementation the size of a block (in bytes) is required to be at least

because we must be able to link a one-block area into the free list. On a 32-bit machine the minimum block size is typically 8 bytes.

  • Constructor and Destructor
  • Acquiring an Area
  • Releasing an Area


Constructor and Destructor

Program gif defines both the constructor and the destructor for the SinglyLinkedPool class. The constructor takes a single argument of type sizet which specifies the desired size of the storage pool in bytes. Initially, the entire storage pool is empty. Therefore, the entire pool constitutes a single area and the free list contains a single entry.

program30601
Program: SinglyLinkedPool Class Constructor and Destructor Definitions

The three member variables of the SinglyLinkedPool class are defined in Program gif . The numberOfBlocks variable records the total number of blocks in the storage pool that are available to be allocated. The pool variable points to the array of blocks that make up the storage pool. Finally, the sentinel variable refers to an extra block that is used as the sentinel for the free list.

The constructor initializes the member variables as follows: The numberOfBlocks is set to [image], where N is the desired size of the storage pool in bytes. The pool member variable is set to point at an array of Blocks of length [image]. The array is itself dynamically allocated using operator new!

The extra block allocated at the end of the array is used as the sentinel for the free list. Therefore, the sentinel member variable is initialized in the constructor as a reference to the last array element. A sentinel is used because it simplifies the linked-list manipulations by eliminating some of the conditional tests that would otherwise be needed.

Initially, the entire pool constitutes a single unallocated area. The entire area is represented by the first block in the pool, pool[0]. Therefore, the length field of the first block is set to numberOfBlocks and the block is linked into the free list by attaching it to the sentinel. Except for the call to operator new to acquire the pool in the first place, the worst-case running time of the constructor is O(1).

The destructor is quite simple. It simply releases the storage pool that was dynamically allocated in the constructor. Notice that it is not necessary that all storage acquired from the pool be released before the destructor is invoked. For example, the programmer may have deliberately chosen not to release the storage or she may have unintentionally forgotten to release the storage (called a memory leak). In any event, after the pool has been finalized, it is a serious error to attempt to dereference a pointer to an area in that pool.


Acquiring an Area

The Acquire function is used to reserve an area in the pool. The code for the Acquire member function of the SinglyLinkedPool class is given in Program gif . The Acquire function takes a single argument that specifies the size of the memory area to be allocated (in bytes). The function returns a pointer to the allocated area of the pool.

program30634
Program: SinglyLinkedPool Class Acquire Member Function Definition

The function begins by calculating the number of blocks required using the formula

I.e., enough storage is set aside to hold the requested number of bytes plus a Header (lines 3-5).

The Acquire function then traverses the linked list of free areas to find a free area that is large enough to satisfy the request (lines 7-13). This is the so-called first-fit allocation strategy : It always allocates storage in the first free area that is large enough to satisfy the request.

An alternative to the first-fit strategy is the best-fit allocation strategy . In the best fit strategy, the Acquire function allocates storage from the free area the size of which matches most closely the requested size. Under certain circumstances, the best-fit strategy may prevent excessive fragmentation of the storage pool. However, the best-fit strategy requires that the entire free list be traversed. Since we are interested in the fastest possible execution time, the first-fit strategy is used here.

If the search for a free area is unsuccessful, a badalloc exception is thrown (lines 14-15). Otherwise, the variable ptr points the free area in which the allocation takes place and the variable prevPtr points to its predecessor in the singly-linked list. If the free area is exactly the correct size, it is simply unlinked from the free list (line 24) and a pointer to the userPart of the area is returned (line 25).

On the other hand, if the free area is larger than needed, the area is split into two areas. The size of the first area is set to the number of blocks requested and the size of the second area is equal to the number of blocks that remain. The second area is then inserted into the free list (lines 16-23). Notice that the area which is unlinked from the free list (line 24) is always equal in size to the required number of blocks.

The running time of the Acquire function is determined by the number of iterations of the loop on lines 9-13. All of the remaining statements in the function require a constant amount of time in the worst-case. The number of iterations of the loop is determined by three factors: the length of the free list, the size of the area requested, and the position in the free list of an area that is large to satisfy the request.

If we make no assumptions about the distribution of the sizes requested and of the pattern of Acquire and Release operations, we cannot say very much about the running time. In the worst case, if there are n blocks in the storage pool, the running time of the Acquire function is O(n).

On the other hand, if we know a priori that all the requests are for the same amount of memory, then we can expect the running time of the Acquire function to be O(1) in the worst case. This is because every single block in the free list is guaranteed to be large enough to satisfy any request. And since we are using the first-fit strategy, the request is satisfied by allocating storage in the first area in the free list.


Releasing an Area

It seems that releasing an area to the free list should be quite simple and fast. E.g., to release an area we might simply insert it at the head of the free list. This could be done in constant time.

However, there is a problem with this: The Acquire function occasionally splits free areas. And if we never coalesce adjacent free areas, the free list will eventually contain a large number of small areas that are each individually too small to satisfy a given request, even though there is sufficient contiguous memory available.

Therefore, the Release function needs to check when an area is freed whether the adjacent areas are already free. However, the problem is this: How do we know where the adjacent areas are? The solution we have adopted is to keep the list of free areas sorted by the starting addresses of the areas.

This means that in order to free an area, it must be inserted in the appropriate place in the linked list. And at the point where the appropriate place to do the insertion has been determined, we can check to see if the area to be freed needs to be merged with an adjacent free area.

Program gif gives the code for the Release function of the SinglyLinkedPool class. This function takes as its lone argument the address of the userPart of an area that was previously obtained from the Acquire function. The Release function begins by determining the block which corresponds to the given area and checking that this block is indeed a part of the memory pool (lines 3-7).

program30674
Program: SinglyLinkedPool Class Release Member Function Definition

The loop on lines 9-15 traverses the linked list of free areas and when it terminates, the following is true: The pointer prevPtr either points to the sentinel or it points at a free area the address of which is less than that of the area to be released. The pointer ptr is either zero or its points to a free area the address of which is greater than that of the area to be released. And prevPtr and ptr always point to adjacent elements of the linked list. Figure gif illustrates this situation.

figure30688
Figure: Using a Singly-Linked, Sorted Free List

The area immediately preceding the area to be freed can itself be either reserved or free. Similarly, the area immediately following the area to be freed may be reserved or free. Figure gif shows two of the four possible situations that can arise. Specifically, in Figure gif (a) both adjacent areas are reserved and in Figure gif (b) both adjacent areas are free.

If the area to be freed immediately precedes a free area, the two areas are combined (lines 16-20). Otherwise, the area is inserted into the free list in front of the area point to by ptr (line 22).

Similarly, if the area to be freed is immediately follows a free area, the two areas are combined (lines 23-27). Otherwise, the are is inserted into the free list following the area pointed to by prevPtr (line 29).

Unfortunately, since the free list is kept sorted, the running time of the Release function is determined by the number of iterations required to find the position in the list at which to do the insertion. In the worst case, this is O(n) where n is the number blocks in the storage pool. In practice, the free list is significantly shorter than n, and the running time varies accordingly.

The combination of keeping the free list sorted by address and the first-fit allocation strategy sometimes leads to a degradation in the performance because the smaller free areas tend to appear near the head of the free list whereas the larger areas are found near the tail of the free list. This is because storage is always allocated in the first area that is large enough and if that area is too large, it is split in two and the left-over area is inserted into the free list. Eventually, many of the areas near the head of the free list are too small to satisfy most requests. Nevertheless, it is necessary to visit those areas every time the free list is traversed.

The use of a minimum block size alleviates partially this bias. I.e., the minimum block size sets the lower bound beyond which areas are not split. For example, in the implementation given the block size is 16 bytes. As a result, any request for storage up to 12 bytes can be satisfied in constant time because the first area in the free list is guaranteed be at least one block in length.


Doubly Linked Free Storage

The singly-linked free list used in the preceding section is kept sorted in order to facilitate the coalescing of adjacent free areas. The only way to determine whether a particular area is free is to see if it appears in the free list. Since we need to examine adjacent areas to determine whether they can be combined, the free list is kept sorted. That way, adjacent areas appear next to one another in the free list.

Unfortunately since the free list must be kept sorted, every time an area is released it must be inserted into the list at the correct position. This means that the running time of the Release operation is O(n) in the worst case, where n is the number of blocks in the storage pool.

The essence of the problem is that we cannot tell by looking at an area whether it is reserved or free. So, the solution must be to record the allocation status of the area in the area itself.

A secondary problem with the use of a singly-linked free list is that, given a pointer to an area, we cannot extract that area from the free list without traversing the list. This is because in order to extract an element from a linked list, we need to know the predecessor of that element. And in a singly-linked list we must search from the head of the list to find the predecessor. The solution is, of course, to use a doubly-linked free list.

Finally, we shall play an interesting algorithmic trick: In order to coalesce adjacent free areas, we need to traverse the free list. However, since we must traverse the free list in the Acquire operation, in order to find a free area of a suitable size, we shall do the coalescing in the Acquire operation and not in the Release operation. While this does increase slightly the running time for Acquire, it means that Release can run in constant time.

Figure gif shows a memory map of a storage pool managed using a doubly-linked list. The reserved areas in Figure gif are exactly the same as those shown in Figure gif . The figure clearly indicates that there are adjacent uncoalesced free areas in the free list. What the figure cannot show is that the areas in the free list are not sorted.

figure30965
Figure: Memory Map of a Doubly-Linked Storage Pool

As before, we implement the storage pool as an array of Blocks. The structure of a Block is shown in Figure gif . A sequence of consecutive, contiguous blocks in the array constitutes an area. Only the first block in each area is used to keep track of the entire area.

figure30976
Figure: DoublyLinkedPool::Block Structure Layout

Notice that we now encode two pieces of information in the block header: A single-bit is used to indicate whether the area is reserved or free and the remaining bits are used to record the length of the area (in blocks). By packing this information in a single word, we have not increased the space overhead associated with reserved areas.

Free areas are linked in a doubly-linked free list. Two pointers are required to accomplish this--prev and next. The effect of the extra pointer is to increase the minimum block size. However, since the pointers occupy space in the pool which would otherwise be unused, we do not require any additional space.

  • Implementation


Implementation

Program gif declares the class DoublyLinkedPool. It is a concrete class derived from the abstract base class StoragePool. The public interface of the class comprises a constructor, the destructor, and the two member functions Acquire and Release. In addition, two private member functions, Unlink and InsertAfter are declared. These abstract common linked list manipulations.

program31248
Program: DoublyLinkedPool Class Definition

The nested structures Header and Block are used to implement the layout shown in Figure gif . The Header structure contains information which appears in the first block of an area, regardless of whether it is reserved or free. In this case, the header comprises two members, status and length, packed into a single word of memory. The first member is a one-bit field of type Status. Status is an enumeration of the values reserved and free. The second field in the header, length, records the length in blocks of the associated area.

The Block structure contains a union. The union overlays an instance of the structure Links, which is used to hold pointers to the next and previous elements of the free list when the block is free, with the userPart of the block which is the space given to the user when the block is reserved.

In this implementation the size of a block (in bytes) is required to be at least

On a 32-bit machine the minimum block size is typically 12 bytes. In the implementation shown in Program gif , the size of the Block structure is set to 16 bytes.

  • Constructor and Destructor
  • Releasing an Area
  • Acquiring an Area


Constructor and Destructor

The constructor and the destructor of the DoublyLinkedPool class are defined in Program gif . The constructor takes as its lone argument an integer which specifies the size of the storage pool in bytes. The constructor is responsible for allocating the memory for the storage pool and for creating the initial free list. Since the storage pool starts out empty, the free list contains initially just a single entry that treats the entire pool as a one large free area.

program31297
Program: DoublyLinkedPool Class Constructor and Destructor Definitions

The constructor must initialize the three member variables, numberOfBlocks, pool and sentinel, which are declared in Program gif . The number of blocks is set to [image], where N is the desired size of the storage pool (in bytes). An array of Blocks of length [image] is dynamically allocated using operator new and the member variable pool is set to point at the array.

The last block in the array is used as the sentinel for the doubly-linked free list. The member variable sentinel is initialized as a reference to the last block in the array. Since the sentinel block is never to be allocated, it is marked reserved. The next and prev pointers of the sentinel both are initialized to point at the sentinel itself. Thus, the free list is initially empty.

The entire pool is initially a single unallocated area. We represent the area by the first block in the pool, pool[0]. The block is marked free and the length field is set to numberOfBlocks. Then, the block is inserted into the doubly-linked free list by calling the private member function InsertAfter. The InsertAfter function can do the insertion in constant time. The implementation is left as an project for the reader (Project gif ). Except for the call to operator new to allocate space for the pool in the first place, the worst-case running time of the constructor is O(1).


Releasing an Area

Program gif gives the code for the Release member function of the DoublyLinkedPool class. This function takes as its lone argument a pointer to the user part of the memory area to be released.

program31331
Program: DoublyLinkedPool Class Release Member Function Definition

The implementation shown is both simple and fast. The Release function makes no attempt to combine adjacent free areas. And since the free list is not sorted, it simply inserts the area to be freed at the front of the free list. This is done by inserting the area into the free list immediately after the sentinel. The private member function InsertAfter is called to accomplish this.

Because the free list is a doubly-linked list, the insertion can be done in constant time. Therefore, the worst-case running time of the Release function is O(1).


Average Running Times

In the previous section, we found the function, [image], which gives the running time of Program gif as a function both of number of inputs, n, and of the actual input values. Suppose instead we are interested in a function [image] which gives the running time on average for n inputs, regardless of the values of those inputs. In other words, if we run Program gif , a large number of times on a selection of random inputs of length n, what will the average running time be?

We can write the sum of the running times given in Table gif in the following form

where [image] is the probability that line 6 of the program is executed. The probability [image] is given by

I.e., [image] is the probability that the [image] array entry, [image], is larger than the maximum of all the preceding array entries, [image].

In order to determine [image], we need to know (or to assume) something about the distribution of input values. For example, if we know a priori that the array passed to the function FindMaximum is ordered from smallest to largest, then we know that [image]. Conversely, if we know that the array is ordered from largest to smallest, then we know that [image].

In the general case, we have no a priori knowledge of the distribution of the values in the input array. In this case, consider the [image] iteration of the loop. In this iteration [image] is compared with the maximum of the i values, [image] preceding it in the array. Line 6 of Program gif is only executed if [image] is the largest of the i+1 values [image]. All things being equal, we can say that this will happen with probability 1/(i+1). Thus

Substituting this expression for [image] in Equation gif and simplifying the result we get

where [image], is the [image] harmonic number .


Acquiring an Area

The Acquire function of the DoublyLinkedPool class is defined in Program gif . This function takes a single integer-valued argument that specifies the size (in bytes) of the area to be allocated. Acquire returns a pointer to the storage if there is sufficient space left in the pool to accommodate the request. Otherwise, a badalloc exception is thrown.

program31350
Program: DoublyLinkedPool Class Acquire Member Function Definition

In order to find a suitable area in which to allocate space, the Acquire function must traverse the free list. In this case, we choose again to use the first-fit allocation strategy. I.e., storage is allocated in the first area that is large enough.

Recall that the Release function given in the preceding section does not combine adjacent free areas. Therefore, it is the responsibility of the Acquire function to do this. The algorithmic trick is this: Adjacent free areas are recombined while the free list is traversed in search of a free area large enough to accommodate the allocation request!

The Acquire function begins by computing the number of blocks required using the formula

I.e., we need enough space for the requested number of bytes plus a Header (lines 3-5).

Then the Acquire function traverses the free list in search of an area that is large enough (lines 7-21). As each area in the free list is visited, the area that immediately follows the given area in memory is examined to see if it too is free and, therefore, if it should be combined with the given area. Since the size of an area is recorded in its header, we can easily find the area which follows it in memory as shown in Figure gif . And since the status of an area (free or reserved) is recorded in the area itself, we can determine whether the following area is free.

figure31374
Figure: Using a Doubly-Linked (Unsorted) Free List

If the area which follows a given area is indeed free, then it must in be the free list! And because the free list is doubly-linked, we can easily extract that area from the free list in constant time. This is what the Unlink function on line 16 does.

When combining a given area with the area that follows it in memory, we obtain an area the size of which is equal to the sum of the sizes of the areas that were combined (line 17). After the combining it is possible that the area that follows the new larger area is also free. Therefore we need to repeat the combining process again. This is what the loop on lines 11-18 does--it keeps combining adjacent free areas until the area that follows is a reserved area (lines 14-15).

The search for a free area to satisfy the Acquire request terminates at the first area that is large enough (lines 19-20). Notice that if the entire free list is traversed all the adjacent areas that can be merged will have been merged. If the free list is traversed and an area is not found that is large enough to satisfy the request, then the request cannot be satisfied because there is insufficient contiguous memory to do so. In this case a badalloc exception is thrown (lines 22-23).

When the free area is larger than needed, it is split in two. The size of the first area is set to the number of blocks requested and the size of the second area is equal to the number of blocks that remain. The second area is then inserted into the free list by calling the the InsertAfter member function (lines 24-31).

In all cases, by the time execution reaches line 32, the free area is exactly the correct size. The area is unlinked from the free list and marked reserved (lines 32-33). Finally, a pointer to the userPart of the area is returned (line 34).

At first glance it might seem that the nested for loops (lines 8 and 11) would result in a worst-case running time of [image] where n is the number of blocks in the storage pool. However, the outer loop traverses the free list while every complete iteration of the inner loop removes one element from the free list. As a result, the worst-case running time for the nested loops is actually only O(n).

The nice thing about this approach is that the asymptotic running time of the Acquire function would be O(n) even if it did not combine adjacent free areas. In particular, this asymptotic bound is the same as for the singly-linked storage pool. On the other hand, the running time of the Release function is O(1) for the doubly-linked pool which is certainly better than the O(n) worst-case running time of the singly-linked pool. Finally, since the free list is not kept sorted, there is not the same tendency for the short areas to accumulate at the head of the free list as there is in the singly-linked pool.


Buddy System for Storage Management

The preceding sections describe two storage pool implementations that both use a linear list to keep track of the free areas. When the singly-linked list is used, the linear list is kept sorted by address; when the doubly-linked list is used, the order of the areas in the list is essentially random.

Each time an area is to be reserved, the free lists are searched in order to find an area that is sufficiently large to satisfy the request. Since there is no direct relationship between the size of an area and its position in the free list, the search has worst-case running time that is O(l), where l is the length of the free list. And in the worst case l is O(n), where n is the number of blocks in the storage pool.

In this section we present a storage pool implementation, called a buddy system , that uses more than one free list. All of the areas in a given free list have the same size and there is a separate free list for each available size of area. As a result a suitable free area can be found quickly.

Given a storage pool of size N bytes, we would require N free lists altogether if we were to place no restriction the allowable size of an area. This is clearly infeasible. Instead we require that N is a power of two, i.e., [image] for some positive integer m. Furthermore, the size of each area in the pool must also be a power of two. As a result, we only need m+1 free lists, since the allowed sizes of an area (in bytes) are

The key feature of a buddy system is that when a request is made for an area of size [image] for some k less than m, we first look in the corresponding free list for an area with the correct size. Notice that if there are no areas of size [image] left, we can obtain one by splitting an area of size [image] in two. And if there are no areas of size [image] left, we can obtain one of those by splitting an area of size [image] in two, and so on.

The two areas obtained when a larger area is split in two are called buddies. Whenever an area is freed, we check to see if its buddy is also free. If an area of size [image] and its buddy are both free, they can be combined into a single area of size [image].

Of course, the user does not always need an an amount of storage that is exactly a power of two. In those situations where it is not, we shall allocate an amount of memory that is the smallest power of two no less than the amount requested.

Figure gif shows a memory map of a storage pool managed using the buddy system. The reserved areas in Figure gif are exactly the same as those shown in Figure gif . Figure gif shows an important characteristic of the buddy pool: An area of size [image] bytes is always aligned on a [image] byte boundary. E.g., all 1KB areas are aligned on 1KB boundaries. I.e., they begin at 0KB, 1KB, 2KB, ...

figure31625
Figure: Memory Map of a Buddy System Storage Pool

Let b be the offset from the start of the pool (in bytes) of an area of size [image]. Then for b to be aligned on a [image] byte boundary means that [image]. In other words, the binary representation of the number b has the form

where [image] is the [image] bit in the representation of b.

If we take the block of size [image] at offset b and split it into two blocks of size [image], the offsets of the two blocks which result are

I.e., the offsets of the buddies of size [image] differ in only the [image] bit position. This gives us a very simple way to determine the position of the buddy of a given area. I.e., given the offset of a buddy of size [image] is b, the offset of the buddy is given by

Fortunately, it is quite simple to compute Equation gif since all that we need to do is toggle the [image] bit of the binary representation b. This can be done using the bitwise exclusive-or operation as the following function definition shows:

unsigned int Buddy (unsigned int b, unsigned int k)
    { return b ^ (1 << (k - 1U)); }

As before, we implement the storage pool as an array of Blocks. The structure of a Block is shown in Figure gif . A sequence of contiguous blocks in the array constitutes and area. This time the size (in bytes) of every area in the pool is an integer power of two. The first block in each area is used to keep track the entire area.

figure31676
Figure: BuddyPool::Block Structure Layout

The structure of the block is quite similar to that used in the implementation of the DoublyLinkedPool class. I.e., the header is comprised of two parts: A single bit which indicates whether the area represented by the block is reserved or free and a field called k which specifies the size of the area. I.e., the size of the block is [image] bytes.

The free lists are implemented as doubly-linked lists. Therefore, a free block contains two pointers, prev and next, which point to the previous and next areas (respectively) in the free list.

  • Implementation


Implementation

The class BuddyPool is declared in Program gif . BuddyPool is a concrete class derived from the abstract base class StoragePool. The public interface includes a constructor, a destructor, and the member functions Acquire and Release. Three additional private member functions, Buddy, Unlink and InsertAfter are declared.

program31910
Program: BuddyPool Class Definition

The nested structures Header and Block implement the layout given in Figure gif . The Header contains fields which are used in the representation of both reserved and free areas. A Header comprises the two members status and k. The former is a one-bit field of type Status. The latter encodes the length of the block as described above.

The Block structure contains a union that overlays an instance of the structure Links with the userPart of the block. If the block is a free block, the contents of the Links structure are used to link the associated area in the appropriate free list. If the block is reserved, the space in the userPart has been allocated to the user.

The minimum size of a block is the same as for the DoublyLinkedPool class, viz.,

which on a 32-bit machine is typically 12 bytes. In the BuddyPool implementation given in Program gif , the size of the Block structure is set to 16 bytes.

  • Constructor and Destructor
  • Acquiring an Area
  • Releasing an Area


Constructor and Destructor

Program gif defines both the constructor and the destructor for the BuddyPool class. The constructor takes an integer which specifies the desired size of the storage pool in bytes. The constructor allocates the memory for the storage pool and initializes the free lists.

program31963
Program: BuddyPool Class Constructor and Destructor Definitions

The BuddyPool class has four member variables, m, numberOfBlocks, pool and sentinel that are initialized by the constructor as follows: The member variable m represents the size of the storage pool. I.e., the size of the pool is [image] bytes. The constructor computes the quantity [image], where n is the size of storage pool requested by calling the function Log2Ceil. gif

The number of blocks required for the storage pool is [image]. The member variable numberOfBlocks is set to this value. An array that contains [image] blocks is dynamically allocated using operator new and the member variable pool is set to point at the array.

Since there are m+1 free lists, the last m+1 blocks are used as the sentinels for the free lists. The member variable sentinel points at the block whose offset in the array is numberOfBlocks. That way, we can access the [image] sentinel by writing sentinel[i]. Each of the m+1 free lists is initially empty. Therefore, the next and prev pointers in each sentinel are set to point at the given sentinel itself (lines 7-11).

The entire pool is initially an unallocated area of size [image]. The area is represented by the first block in the pool, pool[0]. This area is marked free and the k field is set to m. Finally, the block is inserted into the free list for blocks of size [image] which is the list attached to sentinel[m]. The insertion is done using the private member function InsertAfter. Since all the free lists are doubly-linked lists, the InsertAfter can easily do its work in constant time.

Except for the call to operator new to allocate the array of Blocks, the running time of the constructor is dominated by the loop which creates the free lists. As a result, the worst-case running time for the constructor is O(m).


Acquiring an Area

The Acquire member function of the BuddyPool class is defined in Program gif . This function takes an integer-valued argument which specifies the size of storage area required. The function returns a pointer to the region of memory that is allocated.

program32005
Program: BuddyPool Class Acquire Member Function Definition

The size of the area actually allocated by Acquire is [image] bytes where k' is given by

where bytes is the number of bytes of storage requested by the user. I.e., the area must be sufficiently large to hold both the user's data plus the Header used by the storage pool. Of course if k'>m, the request can never be satisfied.

The next step is to find the smallest free area the size of which is at least [image] bytes. Specifically, we find the smallest value of i, [image], such that the free list for areas of size [image] is not empty (lines 5-7). If all free lists for areas of size [image] through [image] are empty, the request cannot be satisfied and a badalloc exception is thrown (lines 8-9).

Having determined the value of i, we remove the first area from the free list of areas of size [image] (lines 11-12). The private member function Unlink is called to remove the area from the doubly-linked free list. To remove a given element from a doubly-linked free list takes constant time.

If i=k', the area has the correct size and we are done. Otherwise, i>k' and the area is too large. If the area is too large, we can split it into two areas (buddies) of size [image]. One of the buddies is retained while the other is inserted into the appropriate free list. After the split if i-1=k', the we are done. Otherwise, the area still to large and it can be split again (lines 13-20).

Eventually, the size of the area is exactly [image] bytes. The area is marked reserved and a pointer to the userPart is returned (lines 21-22).

The running time of the Acquire function is O(i-k'+1). E.g., when we request of block of size [image] and the free list for blocks of size [image] is not empty, then i=k' and the running time is O(1). On the other hand, suppose that we request an area of size [image] but all the free lists are empty except for one entry in the free list of areas of size [image]. In this case i=m and k'=0. Therefore, the worst-case running time is [image], where N is the number of blocks in the storage pool.


Releasing an Area

To release an area of size [image], we first examine its buddy to see if it is reserved or free. If the area and its buddy are both free, the two free areas can be combined into a single area of size [image]. We then repeat the process to release the combined area of size [image]. Eventually we get an area whose buddy is reserved or, if k=m, the area does not have a buddy. When this occurs, we simply insert the area into the appropriate free list. Program gif shows how this algorithm can be implemented.

program32035
Program: BuddyPool Class Release Member Function Definition

The Release member function of the BuddyPool class takes a pointer to the user part of a reserved area and it frees the area as described above. The Release function begins by determining the block which corresponds to the given area and checking that this block is indeed a part of the memory pool (lines 3-7). Then the block is marked free (line 9).

Each iteration of the main loop (lines 11-19) determines whether the buddy of the area to be released is also free and combines the two areas as needed. The buddy of the area is determined by calling the private member function Buddy (line 13). This function takes as its lone argument a reference to the first block of the area whose buddy we seek. The Buddy function returns a reference to the first block of the buddy. (Since the size of the block, k, is found in the block itself, it is not necessary to pass it as a parameter to the Buddy function). If the buddy is found to be reserved, no more combinations are possible and the main loop terminates (lines 14-15).

If the area and its buddy are both free, the buddy is withdrawn from its free list using the private member function Unlink (line 16). Since the free lists are all doubly-linked lists, it is possible to withdraw the buddy from its free list in constant time. After combining the two areas of size [image], the combined area has size [image] and is represented by the buddy with the smaller address (lines 17-18). Eventually, when no more combining is possible, the main loop terminates and we insert the area into the appropriate free list (line 20). This insertion makes use of the private member function InsertAfter which runs in constant time.

The running time of the Release function depends on the number of iterations of the main loop. If the buddy of the area to be released is reserved, then the Release function runs in constant time. In the worst case, there is only one reserved area of size [image] in the storage pool. If we release this area, m iterations of the main loop are required. Therefore, the worst-case running time for the Release function is [image], where N is the number of blocks in the storage pool.

In the preceding analysis (and in the implementation) we have assumed the smallest area has size [image]. However, since every area must contain a Header, the smallest area that will ever occur has size [image]. For example, on a 32-bit machine the size of the header is likely to be four bytes. Since we cannot have an area of size [image] or [image], the corresponding free lists are never used.

It is also quite common to limit the maximum size of an area to a size [image] where m'<m. Consequently, since there are never any blocks of sizes between [image] and [image], the corresponding free lists are never used. Limiting the maximum size of the free lists improves matters slightly, since the worst-case running times for both the Acquire and the Release functions become O(m').


Applications

One of the problems with building storage pool implementations is that it is very difficult to predict how they will behave when subjected to the sequence of Acquire and Release operations generated by a particular application. Furthermore, an implementation that performs well when subjected to the access pattern of one application may perform very poorly when subjected to the access pattern of another. When things become too difficult to analyze, people often turn to simulation for the answer. In this section, we show how to test a storage pool implementation under a simulated load.

A simulated load is a sequence of Acquire and Release operations that is generated by a program whose behavior is supposed to mimic the behavior of a real application. A typical application periodically calls the Acquire function to allocate some memory for its use. The application holds on to this memory for some amount of time and then it calls Release to return the memory to the storage pool.

We shall mimic this behavior with a time-stepped simulation. A time stepped simulation is a program which has the following form:

for (time t = 0; t < timeLimit; ++t)

    Simulate the behavior of the application at time t.
At each point in time, the application performs the following steps:
  1. It releases all the storage areas that were previously scheduled to be freed at this time.
  2. It acquires a storage area of size [image] and schedules the release of that storage area [image] time units from now.
In order to keep track of the storage areas to be released, a priority queue is used (Chapter gif ). The elements of the priority queue record the address of the area to be freed and are keyed using the time at which the area is to be freed.

The values [image] and [image] in step 2 are randomly generated. The random distributions are chosen to mimic the system behavior that we expect.

For example, we may specify that the size [image] (in bytes) is uniformly distributed in the interval [image] and that the time [image] is uniformly distributed in the interval [1,100]. I.e., at each time step, the application allocates between 100 and [image] bytes of storage which it releases after between 1 and 100 time steps.

  • Implementation


Implementation

This section shows how write the simulation described above. Program gif declares the class Event which is used to keep track of the address of an area and the time at which it is to be freed. Therefore, there are two parts to an event, a VoidPtr and a Time.

program32085
Program: Event Class Definition

Since events will be put into a priority queue, the Event class is derived from the Association class which is defined in Section gif . An association is an ordered pair comprised of a key and a value. In the case of the Event class, the key is the time of the event and the value is address of the area to be free. The events in a priority queue are prioritized by their times.

The simulation program is embodied in the function StoragePoolTest defined in Program gif . This function takes two arguments, a reference to the storage pool to be tested and an integer which specifies the number of simulation cycles to execute.

program32098
Program: Simulation Procedure for Exercising a Storage Pool

The variables size and latency are UniformRVs. The class UniformRV is a random number generator defined in Section gif . This class provides a member function called Sample which is used to sample the random number generator. I.e., every time Sample is called, a different (random) result is returned. The random values are uniformly distributed on the interval which is specified in the constructor (lines 3-4).

The StoragePoolTest routine uses a LeftistHeap priority queue (see Section gif ) to keep track of scheduled events (line 5).

The body of the main loop directly implements the steps outlined above. First, any events scheduled for the current time step are withdrawn from the priority queue and the associated storage areas are released (lines 10-19).

Second, the random variable size is sampled to determine the size of area to allocate and the random variable latency is sampled to determine when the allocated area should be deleted. An area of the required size is acquired and an event is scheduled in priority queue that will cause the area to be freed after the appropriate amount of time as elapsed.

The StoragePoolTest function given in Program gif was used by the author to create Figures gif , gif and gif . Figure gif shows the condition of a SinglyLinkedPool after [image] simulation cycles. Figure gif shows the condition of a DoublyLinkedPool and Figure gif shows the condition of a BuddyPool after exactly the same [image] simulation cycles.


Exercises

  1. Consider the memory map shown in Figure gif . The figure suggests that smaller areas (both free and reserved) are more likely to be found at lower address than at higher addresses.
    1. Explain why this phenomenon occurs and why it is undesirable.
    2. Propose a modification to the acquire algorithm that alleviates this effect.
  2. Consider a singly-linked storage pool. Several strategies are possible when searching a free list for an area of a given size:
    first fit
    Select the first area encountered that is large enough to satisfy the request.
    next fit
    This is similar to first fit, except that the free list is treated as a circular list. Each subsequent search begins from the position where the previous search ended.
    best fit
    Select the smallest area that is large enough to satisfy the request.
    worst fit
    Select the largest area as long as it is large enough to satisfy the request.

    1. Devise a scenario which illustrates that next fit can be better than first fit.
    2. Devise a scenario which illustrates that best fit can be better than first fit.
    3. Devise a scenario which illustrates that first fit can be better than best fit.
    4. Under what conditions (if any) does the worst fit scenario make sense.
  3. Show how Program gif can be modified to implement the next fit storage allocation strategy described in Exercise gif . What is the running time of your algorithm?
  4. Show how Program gif can be modified to implement the best fit storage allocation. What is the running time of your algorithm?
  5. Devise optimal algorithms for acquire and release given we know a priori that all the areas acquired from a storage pool will have the same size. What are the running times of your algorithms?
  6. Consider the memory maps shown in Figure gif and Figure gif . When using the SinglyLinkedPool a large block of unused memory is located at one end the pool whereas when using the DoublyLinkedPool the large free area is located at the other end. Explain why this is so.
  7. Show how Program gif can be modified to implement the best fit storage allocation.
    1. What effect does using the best-fit strategy have on the length of the free list in this case?
    2. What is the running time of your algorithm?
  8. Consider the implementations of the SinglyLinkedPool and DoublyLinkedPool classes. In both cases, the sentinel is located immediately following the last block of the storage pool. Explain how the implementations depend on this.
  9. Devise an algorithm that uses bit manipulation operations to compute

    where n is an integer such that [image].

  10. It is possible to implement a buddy pool in which the size of the pool is not a power of two. What modifications to the algorithms are necessary in order to do this?


Projects

  1. Compare experimentally the first-fit, best-fit, next-fit, and worst-fit storage allocation schemes described in Exercise gif by making suitable variations of the SinglyLinkedPool class declared in Program gif . Hint: Use a simulation program such as the one given in Program gif .
  2. Complete the DoublyLinkedPool class declared in Program gif by providing suitable definitions for the following member functions: Unlink and InsertAfter. Write a test program and test your implementation.
  3. Complete the BuddyPool class declared in Program gif by providing suitable definitions for the following member functions: Unlink and InsertAfter. Write a test program and test your implementation.
  4. All three storage pool implementations given in this chapter allocate the memory used for the pool dynamically by using operator new in the pool constructor. Show by implementing suitable constructors for each of the classes that it is possible to allocate the memory used by one storage pool inside another storage pool. E.g., the code sequence
    Pool& p = *new SinglyLinkedPool (8192);
Pool& q = *new (p) SinglyLinkedPool (1024, p);
    creates two pools, p and q. Pool q is allocated in storage acquired from p.
  5. Design and implement a storage pool class that manages a region of memory that is contained entirely within the pool object itself. Because the size of a class must be determined at compile time, this means that the size of the pool is a fixed constant. Write a test program and test your implementation.
  6. Modify the SinglyLinkedPool class declared in Program gif so that instead of allocating memory in the constructor, we pass to the constructor the address and size of a region to be managed. Write a test program and test your implementation.
  7. Find a C++ program that you have written which uses dynamically allocated storage. Replace the storage manager provided by your compiler and operating system by overloading operators new and delete to use one of the storage pool implementations presented in this chapter. Compare the performance of your program when using the different implementations.
  8. Most C++ programs allocate dynamically only a small number of different object types. Consequently, only a small number of distinct sizes of areas need to be acquired.

    Modify the simulation shown in Program gif as follows: In each cycle acquire either four bytes or 32 bytes. Use a suitable random number generator (see Section gif ) so that the smaller block is acquired 75% of the time.

    In addition, suppose that the number of cycles that a block is in use is an exponentially distributed random variable. Assume that the smaller blocks are held on average for 10 cycles and that the larger blocks are held on average for 100 cycles.

    Use the modified simulation program to compare the various storage pool implementations given in this chapter.


About Harmonic Numbers

The series [image] is called the harmonic series , and the summation

gives rise to the series of harmonic numbers , [image], [image], ... As it turns out, harmonic numbers often creep into the analysis of algorithms. Therefore, we should understand a little bit about how they behave.

A remarkable characteristic of harmonic numbers is that, even though as n gets large and the difference between consecutive harmonic numbers gets arbitrarily small ( [image]), the series does not converge! I.e., [image] does not exist. In other words, the summation [image] goes off to infinity, but just barely.

Figure gif helps us to understand the behavior of harmonic numbers. The smooth curve in this figure is the function y=1/x. The descending staircase represents the function [image]. gif I.e., for [image], y=1/i, for

figure605
Figure: Computing Harmonic Numbers

Notice that the area under the staircase between 1 and n for any integer n>1 is given by

Thus, if we can determine the area under the descending staircase in Figure gif , we can determine the values of the harmonic numbers.

As an approximation, consider the area under the smooth curve y=1/x:

Thus, [image] is approximately [image] for n>1.

If we approximate [image] by [image], the error in this approximation is equal to the area between the two curves. In fact, the area between these two curves is such an important quantity that it has its own symbol, [image] , which is called Euler's constant . The following derivation indicates a way in which to compute Euler's constant:

A program to compute Euler's constant on the basis of this derivation is given in Program gif . While this is not necessarily the most accurate or most speedy way to compute Euler's constant, it does give the correct result to six significant digits.

program741
Program: Program to compute

So, with Euler's constant in hand, we can write down an expression for the [image] harmonic number:

where [image] is the error introduced by the fact that [image] is defined as the difference between the curves on the interval [image], but we only need the difference on the interval [1,n]. As it turns out, it can be shown (but not here), that there exists a constant K such that for large enough values of n, [image]. gif

Since the error term is less than 1/n, we can add 1/n to both sides of Equation gif and still have an error which goes to zero as n gets large. Thus, the usual approximation for the harmonic number is

We now return to the question of finding the average running time of Program gif , which finds the largest element of an array. We can now rewrite Equation gif to give


Algorithmic Patterns and Problem Solvers

This chapter presents a number of different algorithmic patterns. Each pattern addresses a category of problems and describes a core solution strategy for that category. Given a problem to be solved, we may find that there are several possible solution strategies. We may also find that only one strategy applies or even that none of them do. A good programmer is one who is proficient at examining the problem to be solved and identifying the appropriate algorithmic technique to use. The following algorithmic patterns are discussed in this chapter:

direct solution strategies
Brute force algorithms and greedy algorithms.
backtracking strategies
Simple backtracking and branch-and-bound algorithms.
top-down solution strategies
Divide-and-conquer algorithms.
bottom-up solution strategies
Dynamic programming.
randomized strategies
Monte Carlo algorithms and simulated annealing.

  • Brute-Force and Greedy Algorithms
  • Backtracking Algorithms
  • Top-Down Algorithms: Divide-and-Conquer
  • Bottom-Up Algorithms: Dynamic
    Programming
  • Randomized Algorithms
  • Exercises
  • Projects


Brute-Force and Greedy Algorithms

In this section we consider two closely related algorithm types--brute-force and greedy. Brute-force algorithms are distinguished not by their structure or form, but by the way in which the problem to be solved is approached. A brute-force algorithm solves a problem in the most simple, direct or obvious way. As a result, such an algorithm can end up doing far more work to solve a given problem than a more clever or sophisticated algorithm might do. On the other hand, a brute-force algorithm is often easier to implement than a more sophisticated one and, because of this simplicity, sometimes it can be more efficient.

Often a problem can be viewed as a sequence of decisions to be made. For example, consider the problem of finding the best way to place electronic components on a circuit board. To solve this problem we must decide where on the board to place each component. Typically, a brute-force algorithm solves such a problem by exhaustively enumerating all the possibilities. I.e., for every decision we consider each possible outcome.

A greedy algorithm is one that makes the sequence of decisions (in some order) such that once a given decision has been made, that decision is never reconsidered. For example, if we use a greedy algorithm to place the components on the circuit board, once a component has been assigned a position it is never again moved. Greedy algorithms can run significantly faster than brute force ones. Unfortunately, it is not always the case that a greedy strategy leads to the correct solution.

  • Example-Counting Change
  • Example-0/1 Knapsack Problem


Example-Counting Change

Consider the problem a cashier solves every time she counts out some amount of currency. The cashier has at her disposal a collection of notes and coins of various denominations and is required to count out a specified sum using the smallest possible number of pieces.

The problem can be expressed mathematically as follows: Let there be n pieces of money (notes or coins), [image], and let [image] be the denomination of [image]. E.g., if [image] is a dime, then [image]. To count out a given sum of money A we find the smallest subset of P, say [image], such that [image].

One way to represent the subset S is to use n variables [image], such that

Given [image] our objective is to minimize

subject to the constraint

  • Brute-Force Algorithm
  • Greedy Algorithm


Brute-Force Algorithm

Since each of the elements of [image] is either a zero or a one, there are [image] possible values for X. A brute-force algorithm to solve this problem finds the best solution by enumerating all the possible values of X.

For each possible value of X we check first if the constraint [image] is satisfied. A value which satisfies the constraint is called a feasible solution . The solution to the problem is the feasible solution which minimizes [image] which is called the objective function .

Since there are [image] possible values of X the running time of a brute-force solution is [image]. The running time needed to determine whether a possible value is a feasible solution is O(n) and the time required to evaluate the objective function is also O(n). Therefore, the running time of the brute-force algorithm is [image].


Greedy Algorithm

A cashier does not really consider all the possible ways in which to count out a given sum of money. Instead, she counts out the required amount beginning with the largest denomination and proceeding to the smallest denomination.

For example, suppose we have ten coins: five pennies, two nickels, two dimes and quarter. I.e., [image]. To count out 32 cents, we start with a quarter, then add a nickel followed by two pennies. This is a greedy strategy because once a coin has been counted out, it is never taken back. Furthermore, the solution obtained is the correct solution because it uses the fewest number of coins.

If we assume that the pieces of money (notes and coins) are sorted by their denomination, the running time for the greedy algorithm is O(n). This is significantly better than that of the brute-force algorithm given above.

Does this greedy algorithm always produce the correct answer? Unfortunately it does not. Consider what happens if we introduce a 15-cent coin. Suppose we are asked to count out 20 cents from the following set of coins: . The greedy algorithm selects 15 followed by five ones--six coins in total. Of course, the correct solution requires only two coins. The solution found by the greedy strategy is a feasible solution, but it does not minimize the objective function.


Example-0/1 Knapsack Problem

The 0/1 knapsack problem is closely related to the change counting problem discussed in the preceding section: We are given a set of n items from which we are to select some number of items to be carried in a knapsack. Each item has both a weight and a profit. The objective is to chose the set of items that fits in the knapsack and maximizes the profit.

Let [image] be the weight of the [image] item, [image] be the profit accrued when the [image] item is carried in the knapsack, and C be the capacity of the knapsack. Let [image] be a variable the value of which is either zero or one. The variable [image] has the value one when the [image] item is carried in the knapsack.

Given [image] and [image], our objective is to maximize

subject to the constraint

Clearly, we can solve this problem by exhaustively enumerating the feasible solutions and selecting the one with the highest profit. However, since there are [image] possible solutions, the running time required for the brute-force solution becomes prohibitive as n gets large.

An alternative is to use a greedy solution strategy which solves the problem by putting items into the knapsack one-by-one. This approach is greedy because once an item has been put into the knapsack, it is never removed.

How do we select the next item to be put into the knapsack? There are several possibilities:

Greedy by Profit
At each step select from the remaining items the one with the highest profit (provided the capacity of the knapsack is not exceeded). This approach tries to maximize the profit by choosing the most profitable items first.
Greedy by Weight
At each step select from the remaining items the one with the least weight (provided the capacity of the knapsack is not exceeded). This approach tries to maximize the profit by putting as many items into the knapsack as possible.
Greedy by Profit Density
At each step select from the remaining items the one with the largest profit density, [image] (provided the capacity of the knapsack is not exceeded). This approach tries to maximize the profit by choosing items with the largest profit per unit of weight.

While all three approaches generate feasible solutions, we cannot guarantee that any of them will always generate the optimal solution. In fact, it is even possible that none of them does! Table gif gives an example where this is the case.

Table: 0/1 Knapsack Problem Example (C=100)
 |  |  |  | greedy by |  |  |
i |  |  |  | profit | weight | density | optimal solution
1 | 100 | 40 | 0.4 | 1 | 0 | 0 | 0
2 | 50 | 35 | 0.7 | 0 | 0 | 1 | 1
3 | 45 | 18 | 0.4 | 0 | 1 | 0 | 1
4 | 20 | 4 | 0.2 | 0 | 1 | 1 | 0
5 | 10 | 10 | 1.0 | 0 | 1 | 1 | 0
6 | 5 | 2 | 0.4 | 0 | 1 | 1 | 1
total weight |  |  |  | 100 | 80 | 85 | 100
total profit |  |  |  | 40 | 34 | 51 | 55

The bottom line about greedy algorithms is this: Before using a greedy algorithm you must make sure that it always gives the correct answer. Fortunately, in many cases this is true.


Backtracking Algorithms

In this section we consider backtracking algorithms . As in the preceding section, we view the problem to be solved as a sequence of decisions. A backtracking algorithm systematically considers all possible outcomes for each decision. In this sense, backtracking algorithms are like the brute-force algorithms discussed in the preceding section. However, backtracking algorithms are distinguished by the way in which the space of possible solutions is explored. Sometimes a backtracking algorithm can detect that an exhaustive search is unnecessary and, therefore, it can perform much better.

  • Example-Balancing Scales
  • Representing the Solution Space
  • Abstract Backtracking Solvers
  • Branch-and-Bound Solvers
  • Example-0/1 Knapsack Problem Again


Example-Balancing Scales

Consider the set of scales shown in Figure gif . Suppose we are given a collection of n weights, [image], and we are required to place all of the weights onto the scales so that they are balanced.

figure32364
Figure: A Set of Scales

The problem can be expressed mathematically as follows: Let [image] represent the pan in which weight [image] is placed such that

The scales are balanced when the sum of the weights in the left pan equals the sum of the weights in the right pan,

Given an arbitrary set of n weights, there is no guarantee that a solution to the problem exists. A solution always exists if, instead of balancing the scales, the goal is to minimize the difference between between the total weights in the left and right pans. Thus, given [image], our objective is to minimize [image] where

subject to the constraint that all the weights are placed on the scales.

Given a set of scales and collection of weights, we might solve the problem by trial-and-error: Place all the weights onto the pans one-by-one. If the scales balance, a solution has been found. If not, remove some number of the weights and place them back on the scales in some other combination. In effect, we search for a solution to the problem by first trying one solution and then backing-up to try another.

Figure gif shows the solution space for the scales balancing problem. In this case the solution space takes the form of a tree: Each node of the tree represents a partial solution to the problem. At the root (node A) no weights have been placed yet and the scales are balanced. Let [image] be the difference between the the sum of the weights currently placed in the left and right pans. Therefore, [image] at node A.

figure32412
Figure: Solution Space for the Scales Balancing Problem

Node B represents the situation in which weight [image] has been placed in the left pan. The difference between the pans is [image]. Conversely, node C represents the situation in which the weight [image] has been placed in the right pan. In this case [image]. The complete solution tree has depth n and [image] leaves. Clearly, the solution is the leaf node having the smallest [image] value.

In this case (as in many others) the solution space is a tree. In order to find the best solution a backtracking algorithm visits all the nodes in the solution space. I.e., it does a tree traversal . Section gif presents the two most important tree traversals--depth-first and breadth-first . Both kinds can be used to implement a backtracking algorithm.


Representing the Solution Space

This section presents an abstract base class for representing the nodes of a solution space. By defining an abstract interface, it is possible to hide the details of the specific problem to be solved from the backtracking algorithm. In so doing, it is possible to implement completely generic backtracking problem solvers.

Although a backtracking algorithm behaves as if it is traversing a solution tree, it is important to realize that it is not necessary to have the entire solution tree constructed at once. Instead, the backtracking algorithm creates and destroys the nodes dynamically as it explores the solution space.

Program gif defines the abstract class called Solution. The Solution class is intended to serve as the base class from which problem-specific classes are derived. Each Solution instance represents a single node in the solution space.

program32621
Program: Solution Class Definition

The Solution class is derived from the Object base class. Consequently, instances of the Solution class can be inserted in the various containers discussed in the preceding chapters. The Solution class adds the following functions to the inherited interface:

IsFeasible
This function returns true if the solution instance is a feasible solution to the given problem. A solution is feasible if it satisfies the problem constraints.
IsComplete
This function returns true if the solution instance represents a complete solution. A solution is complete when all possible decisions have been made.
Objective
This function returns the value of the objective function for the given solution instance.
Bound
This function returns a value that is a lower bound (if it exists) on the objective function for the given solution instance as well as all the solutions that can possibly be derived from that instance. This is a hook provided for to facilitate the implementation of branch-and-bound backtracking which is described in Section gif .
Clone
This function is used to clone the given solution instance. It returns a reference to another, dynamically allocated solution that is identical to the given solution instance.
Successors
This function returns an iterator that enumerates all of the successors (i.e., the children) of the given solution instance. It is assumed that this iterator creates the children of the given node dynamically.


Abstract Backtracking Solvers

The usual way to implement a backtracking algorithm is to write a function or procedure which traverses the solution space. This section presents an alternate, object-oriented approach that is based on the notion of an abstract solver .

Think of a solver as an abstract machine, the sole purpose of which is to search a given solution space for the best possible solution. A machine is an object. Therefore, it makes sense that we represent it as an instance of some class.

Program gif declares the abstract class Solver. The public interface of this class consists of the single function Solve. This function takes as its lone argument a reference to a Solution instance that is the node in the solution space from which to begin the search. The Solve function returns a reference to the to the best solution found.

program32665
Program: Solver Class Definition

Program gif also declares two protected member variables, bestSolution and bestObjective, and two protected member functions, UpdateBest and DoSolve. Since DoSolve is a pure virtual function, its implementation must be given in a derived class. Program gif gives the implementations for Solve and UpdateBest.

program32687
Program: Solver Class Solve and UpdateBest Member Function Definitions

The Solve function does not search the solution space itself. It is the DoSolve routine, which is provided by a derived class, that does the actual searching. The Solve routine merely sets things up for DoSolve. When DoSolve returns it is expected that the bestSolution variable will point to the best solution and that bestObjective will be the value of the objective function for the best solution. In this case, it is assumed that the goal is to minimize the objective function.

The UpdateBest function is meant to be called by the DoSolve routine as it explores the solution space. As each complete solution is encountered, the UpdateBest routine is called to keep track of the solution which minimizes the objective function.

  • Depth-First Solver
  • Breadth-First Solver


Best-Case and Worst-Case Running Times

In Section gif we derived the average running time of Program gif which finds the largest element of an array. In order to do this we had to determine the probability that a certain program statement is executed. To do this, we made an assumption about the average input to the program.

The analysis can be significantly simplified if we simply wish to determine the worst case running time. For Program gif , the worst-case scenario occurs when line 6 is executed in every iteration of the loop. We saw that this corresponds to the case in which the input array is ordered from smallest to largest. In terms of Equation gif , this occurs when [image]. Thus, the worst-case running time is given by

Similarly, the best-case running time occurs when line 6 is never executed. This corresponds to the case in which the input array is ordered from largest to smallest. This occurs when [image] and best-case running time is

In summary we have the following results for the running time of Program gif :


Depth-First Solver

This section presents a backtracking solver that finds the best solution to a given problem by performing depth-first traversal of the solution space. Program gif declares the concrete DepthFirstSolver class. This DepthFirstSolver class simply provides an implementation for the DoSolve routine.

program32715
Program: DepthFirstSolver Class Definition

The DoSolve routine for the DepthFirstSolver class is given in Program gif . Clearly, this routine simply does a complete, depth-first traversal of the solution space. gif Note that the implementation does not depend upon the characteristics of the problem being solved. In this sense the solver is a generic, abstract solver and can be used to solve any problem that has a tree-structured solution space!

program32731
Program: DepthFirstSolver Class DoSolve Member Function Definition

Since the DoSolve routine in Program gif visits all the nodes in the solution space, it is essentially a brute-force algorithm. And because the recursive routine backs up and then tries different alternatives, it is called a backtracking algorithm.


Breadth-First Solver

If we can find the optimal solution by doing a depth-first traversal of the solution space, then we can find the solution with a breadth-first traversal too. As defined in Section gif , a breadth-first traversal of a tree visits the nodes in the order of their depth in the tree. I.e., first the root is visited, then the children of the root are visited, then the grandchildren are visited, and so on.

As shown in Program gif , the BreadthFirstSolver class simply provides an implementation for the DoSolve member function.

program32749
Program: BreadthFirstSolver Class Definition

The body of the DoSolve routine is given in Program gif . This non-recursive, breadth-first traversal algorithm uses a queue to keep track of nodes to be visited. The initial solution is enqueued first. Then the following steps are repeated until the queue is empty:

  1. Dequeue the first solution in the queue.
  2. If the solution is complete, call the UpdateBest routine to keep track of the solution which minimizes the objective function.
  3. Otherwise the solution is not complete. Enqueue all its successors.
Clearly, this algorithm does a complete traversal of the solution space. gif

program32766
Program: BreadthFirstSolver Class DoSolve Member Function Definition


Branch-and-Bound Solvers

The depth-first and breadth-first backtracking algorithms described in the preceding sections both naıvely traverse the entire solution space. However, sometimes we can determine that a given node in the solution space does not lead to the optimal solution--either because the given solution and all its successors are infeasible or because we have already found a solution that is guaranteed to be better than any successor of the given solution. In such cases, the given node and its successors need not be considered. In effect, we can prune the solution tree, thereby reducing the number of solutions to be considered.

For example, consider the scales balancing problem described in Section gif . Consider a partial solution [image] in which we have placed k weights onto the pans ( [image]) and, therefore, n-k weights remain to be placed. The difference between the weights of the left and right pans is given by

and the sum of the weights still to be placed is

Suppose that [image]. I.e., the total weight remaining is less than the difference between the weights in the two pans. Then, the best possible solution that we can obtain without changing the positions of the weights that have already been placed is [image] The quantity [image] is a lower bound on the value of the objective function for all the solutions in the solution tree below the given partial solution [image].

In general, during the traversal of the solution space we may have already found a complete, feasible solution for which the objective function is less than [image]. In that case, there is no point in considering any of the solutions below [image]. I.e., we can prune the subtree rooted at node [image] from the solution tree. A backtracking algorithm that prunes the search space in this manner is called a branch-and-bound algorithm.

  • Depth-First, Branch-and-Bound Solver


Depth-First, Branch-and-Bound Solver

Only a relatively minor modification of the simple, depth-first solver shown in Program gif is needed to transform it into a branch-and-bound solver. Program gif gives the implementation of the DoSolve routine of the DepthFirstBranchAndBoundSolver class. gif

program32802
Program: DepthFirstBranchAndBoundSolver ClassDoSolve Member Function Definition

The only difference between the simple, depth-first solver and the branch-and-bound version is the if statement on lines 11-12. As each node in the solution space is visited two tests are done: First, the IsFeasible routine is called to check whether the given node represents a feasible solution. Next, the Bound routine is called to determine the lower bound on the best possible solution in the given subtree. The second test determines whether this bound is less than the value of the objective function of the best solution already found. The recursive call to explore the subtree is only made if both tests succeed. Otherwise, the subtree of the solution space is pruned.

The degree to which the solution space may be pruned depends strongly on the nature of the problem being solved. In the worst case, no subtrees are pruned and the branch-and-bound routine visits all the nodes in the solution space. The branch-and-bound technique is really just a heuristic --sometimes it works and sometimes it does not.

It is important to understand the trade-off being made: The solution space is being pruned at the added expense of performing the tests as each node is visited. The technique is successful only if the savings which accrue from pruning exceed the additional execution time arising from the tests.


Example-0/1 Knapsack Problem Again

Consider again the 0/1 knapsack problem described in Section gif . We are given a set of n items from which we are to select some number of items to be carried in a knapsack. The solution to the problem has the form [image], where [image] is one if the [image] item is placed in the knapsack and zero otherwise. Each item has both a weight, [image], and a profit, [image]. The goal is to maximize the total profit,

subject to the knapsack capacity constraint

A partial solution to the problem is one in which only the first k items have been considered. I.e., the solution has the form [image], where [image]. The partial solution [image] is feasible if and only if

Clearly if [image] is infeasible, then every possible complete solution containing [image] is also infeasible.

If [image] is feasible, the total profit of any solution containing [image] is bounded by

I.e., the bound is equal the actual profit accrued from the k items already considered plus the sum of the profits of the remaining items.

Clearly, the 0/1 knapsack problem can be solved using a backtracking algorithm. Furthermore, by using Equations gif and gif a branch-and-bound solver can potentially prune the solution space, thereby arriving at the solution more quickly.

For example, consider the 0/1 knapsack problem with n=6 items given in Table gif . There are [image] possible solutions and the solution space contains [image] nodes. The simple DepthFirstSolver given in Program gif visits all 127 nodes and generates all 64 solutions because it does a complete traversal of the solution tree. The BreadthFirstSolver of Program gif behaves similarly. On the other hand, the DepthFirstBranchAndBoundSolver shown in Program gif visits only 67 nodes and generates only 27 complete solutions. In this case, the branch-and-bound technique prunes almost half the nodes from the solution space!


Top-Down Algorithms: Divide-and-Conquer

In this section we discuss a top-down algorithmic paradigm called divide and conquer . To solve a given problem, it is subdivided into one or more subproblems each of which is similar to the given problem. Each of the subproblems is solved independently. Finally, the solutions to the subproblems are combined in order to obtain the solution to the original problem.

Divide-and-conquer algorithms are often implemented using recursion. However, not all recursive functions are divide-and-conquer algorithms. Generally, the subproblems solved by a divide-and-conquer algorithm are non-overlapping.

  • Example-Binary Search
  • Example-Computing Fibonacci Numbers
  • Example-Merge Sorting
  • Running Time of Divide-and-Conquer Algorithms
  • Example-Matrix Multiplication


Example-Binary Search

Consider the problem of finding the position of an item in a sorted list. I.e., given the sorted sequence [image] and an item x, find i (if it exists) such that [image]. The usual solution to this problem is binary search .

Binary search is a divide-and-conquer strategy. The sequence S is split into two subsequences, [image] and [image]. The original problem is split into two subproblems: Find x in [image] or [image]. Of course, since the original list is sorted, we can quickly determine the list in which x must appear. Therefore, we only need to solve one subproblem.

Program gif defines the function BinarySearch which takes four arguments, array, x, i and n. This routine looks for the position in array at which item x is found. Specifically, it considers the following elements of the array:

program32877
Program: Divide-and-Conquer Example--Binary Search

The running time of the algorithm is clearly a function of n, the number of elements to be searched. Although Program gif works correctly for arbitrary values of n, it is much easier to determine the running time if we assume that n is a power of two. In this case, the running time is given by the recurrence

Equation gif is easily solved using repeated substitution:

Setting [image] gives [image].


Example-Computing Fibonacci Numbers

The Fibonacci numbers are given by following recurrence

Section gif presents a recursive function to compute the Fibonacci numbers by implementing directly Equation gif . (See Program gif ). The running time of that program is shown to be [image].

In this section we present a divide-and-conquer style of algorithm for computing Fibonacci numbers. We make use of the following identities

for [image]. (See Exercise gif ). Thus, we can rewrite Equation gif as

Program gif defines the function Fibonacci which implements directly Equation gif . Given n>1 it computes [image] by calling itself recursively to compute [image] and [image] and then combines the two results as required.

program32931
Program: Divide-and-Conquer Example--Computing Fibonacci Numbers

To determine a bound on the running time of the Fibonacci routine in Program gif we assume that T(n) is a non-decreasing function. I.e., [image] for all [image]. Therefore [image]. Although the program works correctly for all values of n, it is convenient to assume that n is a power of 2. In this case, the running time of the routine is upper-bounded by T(n) where

Equation gif is easily solved using repeated substitution:

Thus, T(n)=2n-1=O(n).


Example-Merge Sorting

Sorting algorithms and sorters are covered in detail in Chapter gif . In this section we consider a divide-and-conquer sorting algorithm--merge sort . Given an array of n items in arbitrary order, the objective is to rearrange the elements of the array so that they are ordered from the smallest element to the largest one.

The merge sort algorithm sorts a sequence of length n>1 by splitting it into to subsequences--one of length [image], the other of length [image]. Each subsequence is sorted and then the two sorted sequences are merged into one.

Program gif defines the function MergeSort which takes three arguments, array, i and n. The routine sorts the following n elements:

The MergeSort routine calls itself as well as the Merge routine. The purpose of the Merge routine is to merge two sorted sequences, one of length [image], the other of length [image], into a single sorted sequence of length n. This can easily be done in O(n) time. (See Program gif ).

program32974
Program: Divide-and-Conquer Example--Merge Sorting

The running time of the MergeSort routine depends on the number of items to be sorted, n. Although Program gif works correctly for arbitrary values of n, it is much easier to determine the running time if we assume that n is a power of two. In this case, the running time is given by the recurrence

Equation gif is easily solved using repeated substitution:

Setting [image] gives [image].


Running Time of Divide-and-Conquer Algorithms

A number of divide-and-conquer algorithms are presented in the preceding sections. Because these algorithms have a similar form, the recurrences which give the running times of the algorithms are also similar in form. Table gif summarizes the running times of Programs gif , gif and gif .

Table: Running Times of Divide-and-Conquer Algorithms
program | recurrence | solution
Program gif | T(n)=T(n/2)+O(1) |
Program gif | T(n)=2T(n/2)+O(1) | O(n)
Program gif | T(n)=2T(n/2)+O(n) |

In this section we develop a general recurrence that characterizes the running times of many divide-and-conquer algorithms. Consider the form of a divide-and-conquer algorithm to solve a given problem. Let n be a measure of the size of the problem. Since the divide-and-conquer paradigm is essentially recursive, there must be a base case. I.e., there must be some value of n, say [image], for which the solution to the problem is computed directly. We assume that the worst-case running time for the base case is bounded by a constant.

To solve an arbitrarily large problem using divide-and-conquer, the problem is divided into a number smaller problems, each of which is solved independently. Let a be the number of smaller problems to be solved ( [image], [image]). The size of each of these problems is some fraction of the original problem, typically either [image] or [image] ( [image], [image]).

The solution to the original problem is constructed from the solutions to the smaller problems. The running time required to do this depends on the problem to be solved. In this section we consider polynomial running times. I.e., [image] for some integer [image].

For the assumptions stated above, the running time of a divide-and-conquer algorithm is given by

In order to make it easier to find the solution to Equation gif , we drop the [image] s as well as the [image] from the recurrence. We can also assume (without loss of generality) that [image]. As a result, the recurrence becomes

Finally, we assume that n is a power of b. I.e., [image] for some integer [image]. Consequently, the recurrence formula becomes

We solve Equation gif as follows. Divide both sizes of the recurrence by [image] and then telescope :

Adding Equation gif through Equation gif , substituting T(1)=1 and multiplying both sides by [image] gives

In order to evaluate the summation in Equation gif we must consider three cases:

  • Case 1 ( [image])
  • Case 2 ( [image])
  • Case 3 ( [image])
  • Summary


The Last Axiom

In this section we state the last axiom we need for our detailed model of the C++ virtual machine. This axiom addresses the time required to allocate and to free dynamically allocated storage:

Axiom The time required to allocate a fixed amount of storage from the heap using operator new is a constant, [image]. This time does not include any time required for initialization of the storage. Similarly, the time required to return a fixed amount of storage to the heap using operator delete is a constant, [image]. This time does not include any time spent cleaning up the storage before it is returned to the heap.

By applying Axioms gif and gif , we can determine that the running time of the statement

int* ptr = new int;
is [image]. Similarly, the running time of
delete ptr;
is [image].


Case 1 (#tex2html_wrap_inline68825#)

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Case 1 ( )

In this case, the term [image] falls between zero and one. Consider the infinite geometric series summation:

Since the infinite series summation approaches a finite constant C and since each term in the series is positive, the finite series summation in Equation gif is bounded from above by C:

Substituting this result into Equation gif and making use of the fact that [image], and therefore [image], gives


Case 2 (#tex2html_wrap_inline68839#)

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Case 2 ( )

In this case the term [image] is exactly one. Therefore, the series summation in Equation gif is simply

Substituting this result into Equation gif and making use of the fact that [image] and [image] gives


Case 3 (#tex2html_wrap_inline68849#)

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Case 3 ( )

In this case the term [image] is greater than one and we make use of the general formula for a finite geometric series summation (see Section gif ) to evaluate the summation:

Substituting this result in Equation gif and simplifying gives:


Summary

For many divide-and-conquer algorithms the running time is given by the general recurrence shown in Equation gif . Solutions to the recurrence depend on the relative values of the constants a, b and k. Specifically, the solutions satisfy the following bounds:

Table gif how to apply Equation gif to find the running times of the divide-and-conquer algorithms described in the preceding sections. Comparing the solutions in Table gif with those given in Table gif shows the results obtained using the general formula agree with the analyses done in the preceding sections.

Table: Computing Running Times using Equation gif
program | recurrence | a | b | k | case | solution
Program gif | T(n)=T(n/2)+O(1) | 1 | 2 | 0 |  |
Program gif | T(n)=2T(n/2)+O(1) | 2 | 2 | 0 |  |
Program gif | T(n)=2T(n/2)+O(n) | 2 | 2 | 1 |  |


Example-Matrix Multiplication

Consider the problem of computing the product of two matrices. I.e., given two [image] matrices, A and B, compute the [image] matrix [image], the elements of which are given by

Section gif shows that the direct implementation of Equation gif results in an [image] running time. In this section we show that the use of a divide-and-conquer strategy results in a slightly better asymptotic running time.

To implement a divide-and-conquer algorithm we must break the given problem into several subproblems that are similar to the original one. In this instance we view each of the [image] matrices as a [image] matrix, the elements of which are [image] submatrices. Thus, the original matrix multiplication, [image] can be written as

where each [image], [image] and [image] is an [image] matrix.

From Equation gif we get that the result submatrices can be computed as follows:

Here the symbols + and [image] are taken to mean addition and multiplication (respectively) of [image] matrices.

In order to compute the original [image] matrix multiplication we must compute eight [image] matrix products (divide) followed by four [image] matrix sums (conquer). Since matrix addition is an [image] operation, the total running time for the multiplication operation is given by the recurrence:

Note that Equation gif is an instance of the general recurrence given in Equation gif . In this case, a=8, b=2, and k=2. We can obtain the solution directly from Equation gif . Since [image], the total running time is [image]. But this no better than the original, direct algorithm!

Fortunately, it turns out that one of the eight matrix multiplications is redundant. Consider the following series of seven [image] matrices:

Each equation above has only one multiplication. Ten additions and seven multiplications are required to compute [image] through [image]. Given [image] through [image], we can compute the elements of the product matrix C as follows:

Altogether this approach requires seven [image] matrix multiplications and 18 [image] additions. Therefore, the worst-case running time is given by the following recurrence:

As above, Equation gif is an instance of the general recurrence given in Equation gif . and we obtain the solution directly from Equation gif . In this case, a=7, b=2 and k=2. Therefore, [image] and the total running time is

Note [image]. Consequently, the running time of the divide-and-conquer matrix multiplication strategy is [image] which is better (asymptotically) than the straightforward [image] approach.


Bottom-Up Algorithms: Dynamic Programming

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Bottom-Up Algorithms: Dynamic
Programming

In this section we consider a bottom-up algorithmic paradigm called dynamic programming . In order to solve a given problem, a series of subproblems is solved. The series of subproblems is devised carefully in such a way that each subsequent solution is obtained by combining the solutions to one or more of the subproblems that have already been solved. All intermediate solutions are kept in a table in order to prevent unnecessary duplication of effort.

  • Example-Generalized Fibonacci Numbers
  • Example-Computing Binomial Coefficients
  • Application: Typesetting Problem


Example-Generalized Fibonacci Numbers

Consider the problem of computing the generalized Fibonacci numbers . The generalized Fibonacci numbers of order [image] are given by

Notice that the ``normal'' Fibonacci numbers considered in Section gif are the same as the generalized Fibonacci numbers of order 2.

If we write a recursive function that implements directly Equation gif , we get an algorithm with exponential running time. For example, in Section gif it is shown that the time to compute the second-order Fibonacci numbers is [image].

The problem with the direct recursive implementation is that it does far more work than is needed because it solves the same subproblem many times. For example, to compute [image] it is necessary to compute both [image] and [image]. However, in computing [image] it is also necessary to compute [image], and so on.

An alternative to the top-down recursive implementation is to do the calculation from the bottom up. In order to do this we compute the series of sequences

Notice that we can compute [image] from the information contained in [image] simply by using Equation gif .

Program gif defines the function Fibonacci which takes two integer arguments n and k and computes the [image] Fibonacci number of order k using the approach described above. This algorithm uses an array to represent the series of sequences [image]. As each subsequent Fibonacci number is computed it is added to the end of the array.

program33317
Program: Dynamic Programming Example--Computing Generalized Fibonacci Numbers

The worst-case running time of the Fibonacci routine given in Program gif is a function of both n and k:


Example-Computing Binomial Coefficients

Consider the problem of computing the binomial coefficient

given non-negative integers n and m (see Theorem gif ).

The problem with implementing directly Equation gif is that the factorials grow quickly with increasing n and m. For example, [image]. Therefore, it is not possible to represent n! for [image] using unsigned 32-bit integers. Nevertheless it is possible to represent the binomial coefficients [image] up to n=34 without overflowing. For example, [image].

Consider the following recursive definition of the binomial coefficients:

This formulation does not require the computation of factorials. In fact, the only computation needed is addition.

If we implement Equation gif directly as a recursive function, we get a routine whose running time is given by

which is very similar to Equation gif . In fact, we can show that [image] which (by Equation gif ) is not a very good running time at all! Again the problem with the direct recursive implementation is that it does far more work than is needed because it solves the same subproblem many times.

An alternative to the top-down recursive implementation is to do the calculation from the bottom up. In order to do this we compute the series of sequences

Notice that we can compute [image] from the information contained in [image] simply by using Equation gif . Table gif shows the sequence in tabular form--the [image] row of the table corresponds the sequence [image]. This tabular representation of the binomial coefficients is known as Pascal's triangle . gif

Table: Pascal's Triangle
n |  |  |  |  |  |  |  |
0 | 1 |  |  |  |  |  |  |
1 | 1 | 1 |  |  |  |  |  |
2 | 1 | 2 | 1 |  |  |  |  |
3 | 1 | 3 | 3 | 1 |  |  |  |
4 | 1 | 4 | 6 | 4 | 1 |  |  |
5 | 1 | 5 | 10 | 10 | 5 | 1 |  |
6 | 1 | 6 | 15 | 20 | 15 | 6 | 1 |
7 | 1 | 7 | 21 | 35 | 35 | 21 | 7 | 1

Program gif defines the function Binom which takes two integer arguments n and m and computes the binomial coefficient [image] by computing Pascal's triangle. According to Equation gif , each subsequent row depends only on the preceding row--it is only necessary to keep track of one row of data. The implementation shown uses an array of length n to represent a row of Pascal's triangle. Consequently, instead of a table of size [image], the algorithm gets by with O(n) space. The implementation has been coded carefully so that the computation can be done in place. I.e., the elements of [image] are computed in reverse so that they can be written over the elements of [image] that are no longer needed.

program33428
Program: Dynamic Programming Example--Computing Binomial Coefficients

The worst-case running time of the Binom routine given in Program gif is clearly [image].


Application: Typesetting Problem

Consider the problem of typesetting a paragraph of justified text. A paragraph can be viewed as a sequence of n>0 words, [image]. The objective is to determine how to break the sequence into individual lines of text of the appropriate size. Each word is separated from the next by some amount of space. By stretching or compressing the space between the words, the left and right ends of consecutive lines of text are made to line up. A paragraph looks best when the amount of stretching or compressing is minimized.

We can formulate the problem as follows: Assume that we are given the lengths of the words, [image], and that the desired length of a line is D. Let [image] represent the sequence of words from [image] to [image] (inclusive). I.e.,

for [image].

Let [image] be the sum of the lengths of the words in the sequence [image]. I.e.,

The natural length, for the sequence [image] is the sum of the lengths of the words, [image] plus the normal amount of space between those words. Let s be the normal size of the space between two words. Then the natural length of [image] is [image]. Note, we can also define [image] recursively as follows:

In general, when we typeset the sequence [image] all on a single line, we need to stretch or compress the spaces between the words so that the length of the line is the desired length D. Therefore, the amount of stretching or compressing is given by the difference [image]. However, if the sum of the lengths of the words, [image], is longer than the desired line length D, it is not possible to typeset the sequence on a single line.

Let [image] be the penalty associated with typesetting the sequence [image] on a single line. Then,

This definition is of penalty is consistent with the stated objectives: The penalty increases as the difference between the natural length of the sequence and the desired length increases and the infinite penalty disallows lines that are too long.

Finally, we define the quantity [image] for [image] as the minimum total penalty required to typeset the sequence [image]. In this case, the text may be all on one line or it may be split over more than one line. The quantity [image] is given by

We obtain Equation gif as follows: When i=j there is only one word in the paragraph. The minimum total penalty associated with typesetting the paragraph in this case is just the penalty which results from putting the one word on a single line.

In the general case, there is more than one word in the sequence [image]. In order to determine the optimal way in which to typeset the paragraph we consider the cost of putting the first k words of the sequence on the first line of the paragraph, [image], plus the minimum total cost associated with typesetting the rest of the paragraph [image]. The value of k which minimizes the total cost also specifies where the line break should occur.

  • Example
  • Implementation


Example

Suppose we are given a sequence of n=5 words, [image] having lengths [image], respectively, which are to be typeset in a paragraph of width D=60. Assume that the normal width of an inter-word space is s=10.

We begin by computing the lengths of all the subsequences of W using Equation gif . The lengths of all n(n-1)/2 subsequences of W are tabulated in Table gif .

Table: Typesetting Problem
i |  | j=1 | 2 | 3 | 4 | 5
1 | 10 | 10 | 20 | 30 | 42 | 92
2 | 10 |  | 10 | 20 | 32 | 82
3 | 10 |  |  | 10 | 22 | 72
4 | 12 |  |  |  | 12 | 62
5 | 50 |  |  |  |  | 50

Given [image], D and s, it is a simple matter to apply [image] to obtain the one-line penalties, [image], which measure the amount of stretching or compressing needed to set all the words in a given subsequence on a single line. These are tabulated in Table gif .

Table: Penalties
i | j=1 | 2 | 3 | 4 | 5 | j=1 | 2 | 3 | 4 | 5
1 | 50 | 30 | 10 | 12 |  | 50 | 30 | 10 | 12 | 22
2 |  | 50 | 30 | 8 |  |  | 50 | 30 | 8 | 18
3 |  |  | 50 | 28 |  |  |  | 50 | 28 | 38
4 |  |  |  | 48 |  |  |  |  | 48 | 58
5 |  |  |  |  | 10 |  |  |  |  | 10

Given the one-line penalties [image], we can use Equation gif to find for each subsequence of W the minimum total penalty, [image], associated forming a paragraph from the words in that subsequence. These are tabulated in Table gif .

The [image] entry in Table gif gives the minimum total cost of typesetting the entire paragraph. The value 22 was obtained as follows:

This indicates that the optimal solution is to set words [image], [image], [image] and [image] on the first line of the paragraph and leave [image] by itself on the last line of the paragraph. Figure gif illustrates this result.

figure33553
Figure: Typesetting a Paragraph

This formulation of the typesetting problem seems like overkill. Why not just typeset the lines of text one-by-one, minimizing the penalty for each line as we go? In other words why don't we just use a greedy strategy? Unfortunately, the obvious greedy solution strategy does not work!

For example, the greedy strategy begins by setting the first line of text. To do so it must decide how many words to put on that line. The obvious thing to do is to select the value of k for which [image] is the smallest. From Table gif we see that [image] has the smallest penalty. Therefore, the greedy approach puts three words on the first line as shown in Figure gif .

Since the remaining two words do not both fit on a single line, they are set on separate lines. The total of the penalties for the paragraph typeset using the greedy algorithm is [image]. Clearly, the solution is not optimal (nor is it very pleasing esthetically).


A Simplified Model of the Computer

The detailed model of the computer given in the previous section is based on a number of different timing parameters-- [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], and [image]. While it is true that a model with a large number of parameters is quite flexible and therefore likely to be a good predictor of performance, keeping track of the all of the parameters during the analysis is rather burdensome.

In this section, we present a simplified model which makes the performance analysis easier to do. The cost of using the simplified model is that it is likely to be a less accurate predictor of performance than the detailed model.

Consider the various timing parameters in the detailed model. In a real machine, each of these parameters will be a multiple of the basic clock period of the machine. The clock frequency of a modern computers is typically between 100 and 500 MHz. Therefore, the clock period is typically between 2 and 10 ns. Let the clock period of the machine be T. Then each of the timing parameters can be expressed as an integer multiple of the clock period. E.g., [image], where [image], [image].

The simplified model eliminates all of the arbitrary timing parameters in the detailed model. This is done by making the following two simplifying assumptions:

  • All timing parameters are expressed in units of clock cycles. In effect, T=1.
  • The proportionality constant, k, for all timing parameters is assumed to be the same: k=1.
The effect of these two assumptions is that we no longer need to keep track of the various operations separately. To determine the running time of a program, we simply count the total number of cycles taken.

  • An Example-Geometric Series Summation
  • About Arithmetic Series Summation
  • Example-Geometric Series Summation Again
  • About Geometric Series Summation
  • Example-Computing Powers
  • Example-Geometric Series Summation Yet Again


Implementation

Program gif defines the function Typeset which takes three arguments. The first, l, is an array of n unsigned integers that gives the lengths of the words in the sequence to be typeset. The second, D, specifies the desired paragraph width and the third, s, specifies the normal inter-word space.

program33913
Program: Dynamic Programming Example--Typesetting a Paragraph

The routine first computes the lengths, [image], of all possible subsequences (lines 5-11). This is done by using the dynamic programming paradigm to evaluate the recursive definition of [image] given in Equation gif . The running time for this computation is clearly [image].

The next step computes the one-line penalties [image] as given by Equation gif (lines 12-20). This calculation is a straightforward one and its running time is also [image].

Finally, the minimum total costs, [image], of typesetting each subsequence are determined for all possible subsequences (lines 21-26). Again we make use of the dynamic programming paradigm to evaluate the recursive definition of [image] given in Equation gif . The running time for this computation is [image]. As a result, the overall running time required to determine the best way to typeset a paragraph of n words is [image].


Randomized Algorithms

In this section we discuss algorithms that behave randomly. By this we mean that there is an element of randomness in the way that the algorithm solves a given problem. Of course, if an algorithm is to be of any use, it must find a solution to the problem at hand, so it cannot really be completely random.

Randomized algorithms are said to be methods of last resort. This is because they are used often when no other feasible solution technique is known. For example, randomized methods are used to solve problems for which no closed-form, analytic solution is known. They are also used to solve problems for which the solution space is so large that an exhaustive search is infeasible.

To implement a randomized algorithm we require a source of randomness. The usual source of randomness is a random number generator. Therefore, before presenting randomized algorithms, we first consider the problem of computing random numbers.

  • Generating Random Numbers
  • Random Variables
  • Monte Carlo Methods
  • Simulated Annealing


Generating Random Numbers

In this section we consider the problem of generating a sequence of random numbers on a computer. Specifically, we desire an infinite sequence of statistically independent random numbers uniformly distributed between zero and one. In practice, because the sequence is generated algorithmically using finite-precision arithmetic, it is neither infinite nor truly random. Instead, we say that an algorithm is ``good enough'' if the sequence it generates satisfies almost any statistical test of randomness. Such a sequence is said to be pseudorandom .

The most common algorithms for generating pseudorandom numbers are based on the linear congruential random number generator invented by Lehmer. Given a positive integer m called the modulus and an initial seed value [image] ( [image]), Lehmer's algorithm computes a sequence of integers between 0 and m-1. The elements of the sequence are given by

where a and c are carefully chosen integers such that [image] and [image].

For example, the parameters a=13, c=1, m=16 and [image] produce the sequence

The first m elements of this sequence are distinct and appear to have been drawn at random from the set [image]. However since [image] the sequence is cyclic with period m.

Notice that the elements of the sequence alternate between odd and even integers. This follows directly from Equation gif and the fact that m=16 is a multiple of 2. Similar patterns arise when we consider the elements as binary numbers:

The least significant two bits are cyclic with period four and the least significant three bits are cycle with period eight! (These patterns arise because m=16 is also a multiple of 4 and 8). The existence of such patterns make the sequence less random. This suggests that the best choice for the modulus m is a prime number.

Not all parameter values result in a period of m. For example, changing the multiplier a to 11 produces the sequence

the period of which is only m/2. In general because each subsequent element of the sequence is determined solely from its predecessor and because there are m possible values, the longest possible period is m. Such a generator is called a full period generator.

In practice the increment c is often set to zero. In this case, Equation gif becomes

This is called a multiplicative linear congruential random number generator. (For [image] it is called a mixed linear congruential generator).

In order to prevent the sequence generated by Equation gif from collapsing to zero, the modulus m must be prime and [image] cannot be zero. For example, the parameters a=6, m=13 and [image] produce the sequence

Notice that the first 12 elements of the sequence are distinct. Since a multiplicative congruential generator can never produce a zero, the maximum possible period is m-1. Therefore, this is a full period generator.

As the final step of the process, the elements of the sequence are normalized by division by the modulus:

In so doing, we obtain a sequence of random numbers that fall between zero and one. Specifically, a mixed congruential generator ( [image]) produces numbers in the interval [0,1), whereas a multiplicative congruential generator (c=0) produces numbers in the interval (0,1).

  • The Minimal Standard Random Number Generator
  • Implementation


The Minimal Standard Random Number Generator

A great deal of research has gone into the question of finding an appropriate set of parameters to use in Lehmer's algorithm. A good generator has the following characteristics:

  • It is a full period generator.
  • The generated sequence passes statistical tests of randomness.
  • The generator can be implemented efficiently using 32-bit integer arithmetic.

The choice of modulus depends on the arithmetic precision used to implement the algorithm. A signed 32-bit integer can represent values between [image] and [image]. Fortunately, the quantity [image] is a prime number! gif Therefore, it is an excellent choice for the modulus m.

Because Equation gif is slightly simpler than Equation gif , we choose to implement a multiplicative congruential generator (c=0). The choice of a suitable multiplier is more difficult. However, a popular choice is [image] because it satisfies all three criteria given above: It results in a full period random number generator; the generated sequence passes a wide variety of statistical tests for randomness; and it is possible to compute Equation gif using 32-bit arithmetic without overflow.

The algorithm is derived as follows: First, let [image] and [image]. gif In this case, [image], [image] and r<q.

Next, we rewrite Equation gif as follows:

This somewhat complicated formula can be simplified if we let [image]:

Finally, we make use of the fact that m=aq-r to get

Equation gif has several nice properties: Both [image] and [image] are positive integers between 0 and m-1. Therefore the difference [image] can be represented using a signed 32-bit integer without overflow. Finally, [image] is either a zero or a one. Specifically, it is zero when the sum of the first two terms in Equation gif is negative and it is one when the sum is positive. As a result, it is not necessary to compute [image] --a simple test suffices to determine whether the third term is 0 or m.


Implementation

We now describe the implementation of a random number generator based on Equation gif . Program gif defines the RandomNumberGenerator class. This class has only static members . Because there can only be one instance of a static data member, the implementation of the RandomNumberGenerator class follows the singleton design pattern.

program34003
Program: RandomNumberGenerator Class Definition

Program gif gives the implementation of the RandomNumberGenerator class. The SetSeed function is used to specify the initial seed, [image]. The seed must fall between 0 and m-1. If it does not, an exception is thrown.

program34017
Program: RandomNumberGenerator ClassSetSeed and Next Member Function Definitions

The Next function generates the elements of the random sequence. Each subsequent call returns the next element of the sequence. The implementation follows directly from Equation gif . Notice that the return value is normalized. Therefore, the values computed by the Next function are uniformly distributed on the interval (0,1).


Random Variables

In this section we introduce the notion of an abstract random variable . In this context, a random variable is an object that behaves like a random number generator in that it produces a pseudorandom number sequence. The distribution of the values produced depends on the class of random variable used.

Program gif declares several random variable classes. The abstract RandomVariable class defines the shared interface. A single, pure virtual member function is declared--Sample. Given an instance, say rv, of a concrete class derived from RandomVariable, repeated calls of the form

rv.Sample ();
are expected to return successive elements of a pseudorandom sequence.

program34040
Program: RandomVariable, SimpleRV, UniformRV and ExponentialRV Class Definitions

Program gif also declares three concrete random variable classes:

SimpleRV
This class generates random numbers uniformly distributed in the interval (0,1). (It is merely a wrapper for the RandomNumberGenerator defined in the preceding section).
UniformRV
This class generates random numbers which are uniformly distributed in an arbitrary interval (u,v), where u<v. The parameters u and v are specified in the constructor.
ExponentialRV
This class generates exponentially distributed random numbers with a mean value of [image]. The mean value [image] is specified in the constructor.

  • Implementation


Implementation

Program gif gives the implementations for the three concrete random variable classes declared in Program gif .

program34075
Program: SimpleRV, UniformRV and ExponentialRV Member Function Definitions

The implementation of the SimpleRV class is trivial because the RandomNumberGenerator class generates the desired distribution of random numbers. Consequently, the SimpleRV::Sample function simply calls RandomNumberGenerator::Next.

The UniformRV class is also quite simple. Given that the RandomNumberGenerator class generates a sequence random numbers [image] uniformly distributed on the interval (0,1), the linear transformation

suffices to produce a sequence of random numbers [image] uniformly distributed on the interval (u,v).

The ExponentialRV class generates a sequence of random numbers, [image], exponentially distributed on the interval [image] and having a mean value [image]. The numbers are said to be exponentially distributed because the probability that [image] falls between 0 and z is given by

where [image]. The function p(x) is called the probability density function . Thus,

Notice that [image] is a value between zero and one. Therefore, given a random variable, [image], uniformly distributed between zero and one, we can obtain an exponentially distributed variable [image] as follows:

Note, if [image] is uniformly distributed on (O,1), then so too is [image]. The implementation of the ExponentialRV::Sample function follows directly from Equation gif .


Monte Carlo Methods

In this section we consider a method for solving problems using random numbers. The method exploits the statistical properties of random numbers in order to ensure that the correct result is computed in the same way that a gambling casino sets the betting odds in order to ensure that the ``house'' will always make a profit. For this reason, the problem solving technique is called a Monte Carlo method .

To solve a given problem using a Monte Carlo method we devise an experiment in such a way that the solution to the original problem can be obtained from the experimental results. The experiment typically consists of a series of random trials. A random number generator such as the one given in the preceding section is used to create the series of trials.

The accuracy of the final result usually depends on the number of trials conducted. I.e., the accuracy usually increases with the number of trials. This trade-off between the accuracy of the result and the time taken to compute it is an extremely useful characteristic of Monte Carlo methods. If only an approximate solution is required, then a Monte Carlo method can be very fast.

  • Example-Computing


Example-Computing #tex2html_wrap_inline69477#

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Example-Computing

This section presents a simple, Monte Carlo algorithm to compute the value of [image] from a sequence of random numbers. Consider a square positioned in the x-y plane with its bottom left corner at the origin as shown in Figure gif . The area of the square is [image], where r is the length of its sides. A quarter circle is inscribed within the square. Its radius is r and its center is at the origin of x-y plane. The area of the quarter circle is [image].

figure34129
Figure: Illustration of a Monte Carlo Method for Computing

Suppose we select a large number of points at random inside the square. Some fraction of these points will also lie inside the quarter circle. If the selected points are uniformly distributed, we expect the fraction of points in the quarter circle to be

Therefore by measuring f, we can compute [image]. Program gif shows how this can be done.

program34302
Program: Monte Carlo Program to Compute

The Pi routine uses the RandomNumberGenerator defined to generate (x,y) pairs uniformly distributed on the unit square (r=1). Each point is tested to see if it falls inside the quarter circle. A given point is inside the circle when its distance from the origin, [image] is less than r. In this case since r=1, we simply test whether [image].

How well does Program gif work? When 1000 trials are conducted, 792 points are found to lie inside the circle. This gives the value of 3.168 for [image], which is only 0.8% too large. When [image] trials are conducted, 78535956 points are found to lie inside the circle. In this case, we get [image] which is within 0.005% of the correct value!


Simulated Annealing

Despite its name, simulated annealing has nothing to do either with simulation or annealing. Simulated annealing is a problem solving technique based loosely on the way in which a metal is annealed in order to increase its strength. When a heated metal is cooled very slowly, it freezes into a regular (minimum-energy) crystalline structure.

A simulated annealing algorithm searches for the optimum solution to a given problem in an analogous way. Specifically, it moves about randomly in the solution space looking for a solution that minimizes the value of some objective function. Because it is generated randomly, a given move may cause the objective function to increase, to decrease or to remain unchanged.

A simulated annealing algorithm always accepts moves that decrease the value of the objective function. Moves that increase the value of the objective function are accepted with probability

where [image] is the change in the value of the objective function and T is a control parameter called the temperature . I.e., a random number generator that generates numbers distributed uniformly on the interval (0,1) is sampled, and if the sample is less than p, the move is accepted.

By analogy with the physical process, the temperature T is initially high. Therefore, the probability of accepting a move that increases the objective function is initially high. The temperature is gradually decreased as the search progresses. I.e., the system is cooled slowly. In the end, the probability of accepting a move that increases the objective function becomes vanishingly small. In general, the temperature is lowered in accordance with an annealing schedule .

The most commonly used annealing schedule is called exponential cooling . Exponential cooling begins at some initial temperature, [image], and decreases the temperature in steps according to [image] where [image]. Typically, a fixed number of moves must be accepted at each temperature before proceeding to the next. The algorithm terminates either when the temperature reaches some final value, [image], or when some other stopping criterion has been met.

The choice of suitable values for [image], [image] and [image] is highly problem-dependent. However, empirical evidence suggests that a good value for [image] is 0.95 and that [image] should be chosen so that the initial acceptance probability is 0.8. The search is terminated typically after some fixed, total number of solutions have been considered.

Finally, there is the question of selecting the initial solution from which to begin the search. A key requirement is that it be generated quickly. Therefore, the initial solution is generated typically at random. However, sometimes the initial solution can be generated by some other means such as with a greedy algorithm.

  • Example-Balancing Scales


An Example-Geometric Series Summation

In this section we consider the running time of a program to compute the following geometric series summation . I.e., given a value x and non-negative integer n, we wish to compute the summation

An algorithm to compute this summation is given in Program gif .

program814
Program: Program to compute

Table gif gives the running time, as predicted by the simplified model, for each of the executable statements in Program gif .

Table: Computing the running time of Program gif
statement | time
3 | 2
4a | 2
4b | 3(n+2)
4c | 4(n+1)
6 | 2(n+1)
7a | 2(n+1)
7b |
7c |
8 |
9 | 4(n+1)
10 | 2
TOTAL |

In order to calculate the total cycle counts, we need to evaluate the two series summations [image] and [image]. Both of these are arithmetic series summations . In the next section we show that the sum of the series [image] is n(n+1)/2. Using this result we can sum the cycle counts given in Table gif to arrive at the total running time of [image] cycles.


Example-Balancing Scales

Consider again the scales balancing problem described in Section gif . I.e., we are given a set of n weights, [image], which are to be placed on a pair of scales in the way that minimizes the difference between the total weight in each pan. Feasible solution to the problem all have the form [image], where

To solve this problem using simulated annealing, we need a strategy for generating random moves. The move generator should make small, random changes to the current solution and it must ensure that all possible solutions can be reached. A simple approach is to use the formula

where [image] is the initial solution, [image] is a new solution, [image] is a sequence of zeroes and ones generated randomly, and [image] denotes elementwise addition modulo two.


Exercises

  1. Consider the greedy strategy for counting out change given in Section gif . Let [image] be the set of available denominations. E.g., the set [image] represents the denominations of the commonly circulated Canadian coins. What condition(s) must the set of denominations satisfy to ensure the greedy algorithm always finds an optimal solution?
  2. Devise a greedy algorithm to solve optimally the scales balancing problem described in Section gif .
    1. Does your algorithm always find the optimal solution?
    2. What is the running time of your algorithm?
  3. Consider the following 0/1-knapsack problem:

    i
    1 | 10 | 10
    2 | 6 | 6
    3 | 3 | 4
    4 | 8 | 9
    5 | 1 | 3
    C=18

    1. Solve the problem using the greedy by profit, greedy by weight and greedy by profit density strategies.
    2. What is the optimal solution?
  4. Consider the breadth-first solver shown in Program gif . Suppose we replace the queue (line 3) with a priority queue.
    1. How should the solutions in the priority queue be prioritized?
    2. What possible benefit might there be from using a priority queue rather than a FIFO queue?
  5. Repeat Exercise gif , but this time consider what happens if we replace the queue with a LIFO stack.
  6. Repeat Exercises gif and gif , but this time consider a branch-and-bound breadth-first solver.
  7. (This question should be attempted after reading Chapter gif ). For some problems the solution space is more naturally a graph rather than a tree.
    1. What problem arises if we use the DepthFirstSolver given in Program gif to explore a search space that is not a tree.
    2. Modify the DepthFirstSolver so that it explores a solution space that is not a tree. Hint: See Program gif .
    3. What problem arises if we use the BreadthFirstSolver given in Program gif to explore a search space that is not a tree.
    4. Modify the BreadthFirstSolver so that it explores a solution space that is not a tree. Hint: See Program gif .
  8. Devise a backtracking algorithm to solve the N-queens problem : Given an [image] chess board, find a way to place N queens on the board in such a way that no queen can take another.
  9. Consider a binary search tree that contains n keys, [image], [image], ..., [image], at depths [image], [image], ..., [image], respectively. Suppose the tree will be subjected to a large number of Find operations. Let [image] be the probability that we access key [image]. Suppose we know a priori all the access probabilities. Then we can say that the optimal binary search tree is the tree which minimizes the quantity

    1. Devise a dynamic programming algorithm that, given the access probabilities, determines the optimal binary search tree.
    2. What is the running time of your algorithm?

    Hint: Let [image] be the cost of the optimal binary search tree that contains the set of keys [image] where [image]. Show that

  10. Consider the typesetting problem discussed in Section gif . The objective is to determine how to break a given sequence of words into lines of text of the appropriate size. This was done either by stretching or compressing the space between the words. Explain why the greedy strategy always finds the optimal solution if we stretch but do not compress the space between words.
  11. Consider two complex numbers, a+bi and c+di. Show that we can compute the product (ac-bd)+(ad+bc)i with only three multiplications.
  12. Devise a divide-and-conquer strategy to find the root of a polynomial. E.g., given a polynomial such as [image], and an interval [u,v], such that [image] such that [image], [image] and [image], [image], find r.
  13. Devise an algorithm to compute a normally distributed random variable. A normal distribution is complete defined by its mean and standard deviation. The probability density function for a normal distribution is

    where [image] is the mean and [image] is the standard deviation of the distribution. Hint: Consider the central limit theorem .

  14. Devise an algorithm to compute a geometrically distributed random variable. A geometrically distributed random variable is an integer in the interval [image] given by the probability density function

    where [image] is the mean of the distribution.

    Hint: Use the fact [image], where Z is an exponentially distributed random variable with mean [image].

  15. Do Exercise gif .


Projects

  1. Design and implement a class derived from the abstract Solution class defined in Program gif which represents the nodes of the solution space of a 0/1-knapsack problem described in Section gif .

    Devise a suitable representation for the state of a node and then implement the following member functions IsFeasible, IsComplete, Objective, Bound, Clone and Successors. Note, the Successors function requires an iterator which enumerates all the successors of a given node.

    1. Use your class with the DepthFirstSolver defined in Program gif to solve the problem given in Table gif .
    2. Use your class with the BreadthFirstSolver defined in Program gif to solve the problem given in Table gif .
    3. Use your class with the DepthFirstBranchAndBoundSolver defined in Program gif to solve the problem given in Table gif .
  2. Do Project gif for the change counting problem described in Section gif .
  3. Do Project gif for the scales balancing problem described in Section gif .
  4. Do Project gif for the N-queens problem described in Exercise gif .
  5. Design and implement a GreedySolver class, along the lines of the DepthFirstSolver and BreadthFirstSolver classes, that conducts a greedy search of the solution space. To do this you will have to add a member function to the abstract Solution class:
    class GreedySolution : public Solution
{
    virtual Solution& GreedySuccessor () const = 0;
};
  6. Design and implement a SimulatedAnnealingSolver class, along the lines of the DepthFirstSolver and BreadthFirstSolver classes, that implements the simulated annealing strategy described in Section gif To do this you will have to add a member function to the abstract Solution class:
    class SimulatedAnnealingSolution : public Solution
{
    virtual Solution& RandomSuccessor () const = 0;
};
  7. Design and implement a dynamic programming algorithm to solve the change counting problem. Your algorithm should always find the optimal solution--even when the greedy algorithm fails.
  8. Consider the divide-and-conquer strategy for matrix multiplication described in Section gif .
    1. Rewrite the implementation of the multiplication operator, operator*, of the Matrix<T> class declared in Program gif .
    2. Compare the running time of your implementation with the [image] algorithm given in Program gif .
  9. Consider random number generator that generates random numbers uniformly distributed between zero and one. Such a generator produces a sequence of random numbers [image]. A common test of randomness evaluates the correlation between consecutive pairs of numbers in the sequence. One way to do this is to plot on a graph the points

    1. Write a program to compute the first 1000 pairs of numbers generated using the UniformRV declared in Programs gif and gif .
    2. What conclusions can you draw from your results?


Sorting Algorithms and Sorters

  • Basics
  • Sorting and Sorters
  • Insertion Sorting
  • Exchange Sorting
  • Selection Sorting
  • Merge Sorting
  • A Lower Bound on Sorting
  • Distribution Sorting
  • Performance Data
  • Exercises
  • Projects


Basics

Consider an arbitrary sequence [image] comprised of of [image] elements drawn from a some universal set U. The goal of sorting is to rearrange the elements of S to produce a new sequence, say S', in which the elements of S appear in order.

But what does it mean for the elements of S' to be in order? We shall assume that there is a relation, <, defined over the universe U. The relation < must be a total order, which is defined as follows:

Definition A total order is a relation, say <, defined on the elements of some universal set U with the following properties:
  1. For all pairs of elements [image], exactly one of the following is true: i<j, i=j, or j<i.

    (All elements are commensurate ).

  2. For all triples [image], [image].

    (The relation < is transitive ).

In order to sort the elements of the sequence S, we determine the permutation [image] of the elements of S such that

In practice, we are not interested in the permutation P, per se. Instead, our objective is to compute the sorted sequence [image] in which [image] for [image].

Sometimes the sequence to be sorted, S, contains duplicates. I.e., [image] such that [image]. In general when a sequence that contains duplicates is sorted, there is no guarantee that the duplicated elements retain their relative positions. I.e., [image] could appear either before or after [image] in the sorted sequence S'. If duplicates retain their relative positions in the sorted sequence the sort is said to be stable . In order for [image] and [image] to retain their relative order in the sorted sequence, we require that [image] precedes [image] in S'. Therefore, the sort is stable if [image].


Sorting and Sorters

The traditional way to implement a sorting algorithm is to write a function or procedure that sorts an array of data. This chapter presents an alternate, object-oriented approach that is based on the notion of an abstract sorter .

Think of a sorter as an abstract machine, the sole purpose of which is to sort arrays of data. A machine is an object. Therefore, it makes sense that we represent it as an instance of some class. The machine sorts data. Therefore, the class will have a member function, say Sort, which sorts an array of data.

Program gif declares the abstract class template Sorter<T>. The public interface of this class consists of the single function Sort. This function takes as its lone argument a reference to an Array<T> instance and it sorts the data contained therein. We use the Array<T> class defined in Section gif to represent the data to be sorted because all array subscript operations are bounds-checked and because we can determine the length of the array by calling the Length member function of the Array<T> class.

program34553
Program: Sorter Class Definition

Program gif also declares the member variable n, the static member function Swap, and the pure virtual function DoSort. Since DoSort is a pure virtual function, an implementation must be given in a derived class. Program gif gives the implementations for Sort and Swap.

program34574
Program: Sorter<T> class Swap and Sort Member Function Definitions

The Sort function does not sort the data itself. It is the DoSort routine, which is provided by a derived class, that does the actual sorting. The Sort routine merely sets things up for DoSort.

The Sort function takes a reference to an Array<T> instance which allows arbitrary array subscript ranges. E.g., the first element of an array a is the one at position a[a.Base()]. However, in most cases it is easier to write the sorting algorithm for a fixed lower bound of, say, zero, For this reason, the Sort function sets the array base to zero. Of course, the caller does not expect the array base to be changed. Therefore, the Sort routine restores the original array base before it returns.

The Swap function is used to implement most of the sorting algorithms presented in this chapter. The swap function takes two references to objects of some type T, and exchanges the contents of these two objects. The exchange is done as a sequence of three assignments. Therefore, if a T instance can be assigned in constant time, the Swap routine runs in constant time. gif

  • Sorter Class Hierarchy


Sorter Class Hierarchy

This chapter describes nine different sorting algorithms. These are organized into the following five categories:

  • insertion sorts
  • exchange sorts
  • selection sorts
  • merge sorts
  • distribution sorts .
As shown in Figure gif , the sorter classes have been arranged in a class hierarchy that reflects this classification scheme.

figure34610
Figure: Sorter Class Hierarchy

Program gif defines five abstract sorter class templates: InsertionSorter<T>, ExchangeSorter<T>, SelectionSorter<T>, MergeSorter<T> and DistributionSorter<T>. Notice that these classes are all trivial (i.e., empty) extensions of the Sorter<T> abstract base class.

program34626
Program: Classes of Sorters


Insertion Sorting

The first class of sorting algorithm that we consider comprises algorithms that sort by insertion . An algorithm that sorts by insertion takes the initial, unsorted sequence, [image], and computes a series of sorted sequences [image], as follows:

  1. The first sequence in the series, [image] is the empty sequence. I.e., [image].
  2. Given a sequence [image] in the series, for [image], the next sequence in the series, [image], is obtained by inserting the [image] element of the unsorted sequence [image] into the correct position in [image].
Each sequence [image], [image], contains the first i elements of the unsorted sequence S. Therefore, the final sequence in the series, [image], is the sorted sequence we seek. I.e., [image].

Figure gif illustrates the insertion sorting algorithm. The figure shows the progression of the insertion sorting algorithm as it sorts an array of ten integers. The array is sorted in place . I.e., the initial unsorted sequence, S, and the series of sorted sequences, [image], occupy the same array.

figure34651
Figure: Insertion Sorting

In the [image] step, the element at position i in the array is inserted into the sorted sequence [image] which occupies array positions 0 to (i-1). After this is done, array positions 0 to i contain the i+1 elements of [image]. Array positions (i+1) to (n-1) contain the remaining n-i-1 elements of the unsorted sequence S.

As shown in Figure gif , the first step (i=0) is trivial--inserting an element into the empty list involves no work. Altogether, n-1 non-trivial insertions are required to sort a list of n elements.

  • Straight Insertion Sort
  • Average Running Time
  • Binary Insertion Sort


Straight Insertion Sort

The key step of any insertion sorting algorithm involves the insertion of an item into a sorted sequence. There are two aspects to an insertion--finding the correct position in the sequence at which to insert the new element and moving all the elements over to make room for the new one.

This section presents the straight insertion sorting algorithm. Straight insertion sorting uses a linear search to locate the position at which the next element is to be inserted.

  • Implementation


Implementation

Program gif defines the class template StraightInsertionSorter<T>. This class simply provides an implementation for the DoSort routine. DoSort takes one argument--a reference to an Array<T> instance. The array initially contains the data to be sorted. When the function returns, the contents of the array are sorted.

program35297
Program: StraightInsertionSorter<T> Class DoSort Member Function Definition

In order to determine the running time of the DoSort routine given in Program gif , we need to determine the number of iterations of the inner loop (lines 12-14). The number of iterations of the inner loop in the [image] iteration of the outer loop depends on the positions of the values in the array. In the best case, the value in position i of the array is larger than that in position i-1 and zero iterations of the inner loop are done. In this case, the running time for insertion sort is O(n). Notice that the best case performance occurs when we sort an array that is already sorted!

In the worst case, i iterations of the inner loop are required in the [image] iteration of the outer loop. This occurs when the value in position i of the array is smaller than the values at positions 0 through i-1. Therefore, the worst case arises when we sort an array in which the elements are initially sorted in reverse. In this case the running time for insertion sort is [image].


About Arithmetic Series Summation

The series, [image], is an arithmetic series and the summation

is called the arithmetic series summation .

The summation can be solved as follows: First, we make the simple variable substitution i=n-j:

Note that the term in the first summation in Equation gif is independent of j. Also, the second summation is identical to the left hand side. Rearranging Equation gif , and simplifying gives

There is, of course, a simpler way to arrive this answer. Consider the series, [image]. and suppose n is even. The sum of the first and last element is n+1. So too is the sum of the second and second-last element, and the third and third-last element, etc., and there are n/2 such pairs. Therefore, [image].

And if n is odd, then [image], where n-1 is even. So we can use the previous result for [image] to get [image].


Average Running Time

The best case running time of insertion sorting is O(n) but the worst-case running time is [image]. Therefore, we might suspect that the average running time falls somewhere in between. In order to determine it, we must define more precisely what we mean by the average running time. A simple definition of average running time is to say that it is the running time needed to sort the average sequence. But what is the average sequence?

The usual way to determine the average running time of a sorting algorithm is to consider only sequences that contain no duplicates. Since every sorted sequence of length n is simply a permutation of an unsorted one, we can represent every such sequence by a permutation of the sequence [image]. When computing the average running time, we assume that every permutation is equally likely. Therefore, the average running time of a sorting algorithm is the running time averaged over all permutations of the sequence S.

Consider a permutation [image] of the sequence S. An inversion in P consists of two elements, say [image] and [image], such that [image] but i<j. I.e., an inversion in P is a pair of elements that are in the wrong order. For example, the permutation [image] contains three inversions--(4,3), (4,2), and (3,2). The following theorem tells us how many inversions we can expect in the average sequence:

Theorem The average number of inversions in a permutation of n distinct elements is n(n-1)/4.

extbfProof Let S be an arbitrary sequence of n distinct elements and let [image] be the same sequence, but in reverse.

E.g., if [image], then [image].

Consider any pair of distinct elements in S, say [image] and [image] where [image]. There are two distinct possibilities: Either [image], in which case [image] is an inversion is [image]; or [image], in which case [image] is an inversion is S. Therefore, every pair contributes exactly one inversion either to S or to [image].

The total number of pairs in S is [image]. Since every such pair contributes an inversion either to S or to [image], we expect on average that half of the inversions will appear in S. Therefore, the average number of inversions in a sequence of n distinct elements is n(n-1)/4.

What do inversions have to do with sorting? As a list is sorted, inversions are removed. In fact, since the inner loop of the insertion sort routine swaps adjacent array elements, inversions are removed one at a time! Since a swap takes constant time, and since the average number of inversions is n(n-1)/4, the average running time for the insertion sort routine is [image].


Binary Insertion Sort

The straight insertion algorithm presented in the preceding section does a linear search to find the position in which to do the insertion. However, since the element is inserted into a sequence that is already sorted, we can use a binary search instead of a linear search. Whereas a linear search requires O(n) comparisons in the worst case, a binary search only requires [image] comparisons. Therefore, if the cost of a comparison is significant, the binary search may be preferred.

Program gif defines the DoSort routine of the BinaryInsertionSorter<T> class. The framework of this routine is essentially the same as that of the StraightInsertionSorter<T> class.

program35333
Program: BinaryInsertionSorter<T> Class DoSort Member Function Definition

Exactly, n-1 iterations of the outer loop are done (lines 11-26). In each iteration, a binary search search is done to determine the position at which to do the insertion (lines 13-23). In the [image] iteration of the outer loop, the binary search considers array positions 0 to i (for [image]). The running time for the binary search in the [image] iteration is [image]. Once the correct position is found, at most i swaps are needed to insert the element in its place.

The worst-case running time of the binary insertion sort is dominated by the i swaps needed to do the insertion. Therefore, the worst-case running time is [image]. Furthermore, since the algorithm only swaps adjacent array elements, the average running time is also [image] (see Section gif ). Asymptotically, the binary insertion sort is no better than straight insertion.

However, the binary insertion sort does fewer array element comparisons than insertion sort. In the [image] iteration of the outer loop, the binary search requires [image] comparisons, for [image]. Therefore, the total number of comparisons is

(This result follows directly from Theorem gif ).

The number of comparisons required by the straight insertion sort is [image] in the worst case as well as on average. Therefore on average, the binary insertion sort uses fewer comparisons than straight insertion sort. On the other hand, the previous section shows that in the best case the running time for straight insertion is O(n). Since the binary insertion sort routine always does the binary search, its best case running time is [image]. Table gif summarizes the asymptotic running times for the two insertion sorts.

Table: Running Times for Insertion Sorting
 | running time
algorithm | best case | average case | worst case
straight insertion sort | O(n) |  |
binary insertion sort |  |  |


Exchange Sorting

The second class of sorting algorithm that we consider comprises algorithms that sort by exchanging pairs of items until the sequence is sorted. In general, an algorithm may exchange adjacent elements as well as widely separated ones.

In fact, since the insertion sorts considered in the preceding section accomplish the insertion by swapping adjacent elements, insertion sorting can be considered as a kind of exchange sort. The reason for creating a separate category for insertion sorts is that the essence of those algorithms is insertion into a sorted list. On the other hand, an exchange sort does not necessarily make use of such a sorted list.

  • Bubble Sort
  • Quicksort
  • Running Time Analysis
  • Average Running Time
  • Selecting the Pivot


Bubble Sort

The simplest and, perhaps, the best known of the exchange sorts is the bubble sort . gif Figure gif shows the operation of bubble sort.

figure35378
Figure: Bubble Sorting

To sort the sequence [image], bubble sort makes n-1 passes through the data. In each pass, adjacent elements are compared and swapped if necessary. First, [image] and [image] are compared; next, [image] and [image]; and so on.

Notice that after the first pass through the data, the largest element in the sequence has bubbled up into the last array position. In general, after k passes through the data, the last k elements of the array are correct and need not be considered any longer. In this regard the bubble sort differs from the insertion sort algorithms--the sorted subsequence of k elements is never modified (by an insertion).

Figure gif also shows that while n-1 passes through the data are required to guarantee that the list is sorted in the end, it is possible for the list to become sorted much earlier! When no exchanges at all are made in a given pass, then the array is sorted and no additional passes are required. A minor algorithmic modification would be to count the exchanges made in a pass, and to terminate the sort when none are made.

Program gif defines the BubbleSorter<T> class template. This class simply provides an implementation for the DoSort routine. The DoSort routine takes a reference to an Array<T> instance and sorts its elements in place. The implementation makes use of the Swap routine described in Section gif .

program36562
Program: BubbleSorter<T> Class DoSort Member Function Definition

The outer loop (lines 11-14) is done for [image]. That makes n-1 iterations in total. During the [image] iteration of the outer loop, exactly i-1 iterations of the inner loop are done (lines 12-14). Therefore, the number of iterations of the inner loop, summed over all the passes of the outer loop is

Consequently, the running time of bubble sort is [image].

The body of the inner loop compares adjacent array elements and swaps them if necessary (lines 13-14). This takes at most a constant amount of time. Of course, the algorithm will run slightly faster when no swapping is needed. For example, this occurs if the array is already sorted to begin with. In the worst case, it is necessary to swap in every iteration of the inner loop. This occurs when the array is sorted initially in reverse order. Since only adjacent elements are swapped, bubble sort removes inversions one at time. Therefore, the average number of swaps required is [image]. Nevertheless, the running time of bubble sort is always [image].


Quicksort

The second exchange sort we consider is the quicksort algorithm. Quicksort is a divide-and-conquer style algorithm. A divide-and-conquer algorithm solves a given problem by splitting it into two or more smaller subproblems, recursively solving each of the subproblems, and then combining the solutions to the smaller problems to obtain a solution to the original one.

To sort the sequence [image], quicksort performs the following steps:

  1. Select one of the elements of S. The selected element, p, is called the pivot .
  2. Remove p from S and then partition the remaining elements of S into two distinct sequences, L and G, such that every element in L is less than or equal to the pivot and every element in G is greater than or equal to the pivot. In general, both L and G are unsorted.
  3. Rearrange the elements of the sequence as follows:

    Notice that the pivot is now in the position in which it belongs in the sorted sequence, since all the elements to the left of the pivot are less than or equal to the pivot and all the elements to the right are greater than or equal to it.

  4. Recursively quicksort the unsorted sequences L and G.

The first step of the algorithm is a crucial one. We have not specified how to select the pivot. Fortunately, the sorting algorithm works no matter which element is chosen to be the pivot. However, the pivot selection affects directly the running time of the algorithm. If we choose poorly the running time will be poor.

Figure gif illustrates the detailed operation of quicksort as it sorts the sequence [image]. To begin the sort, we select a pivot. In this example, the value 4 in the last array position is chosen. Next, the remaining elements are partitioned into two sequences, one which contains values less than or equal to 4 ( [image]) and one which contains values greater than or equal to 4 ( [image]). Notice that the partitioning is accomplished by exchanging elements. This is why quicksort is considered to be an exchange sort.

figure36594
Figure: ``Quick'' Sorting

After the partitioning, the pivot is inserted between the two sequences. This is called restoring the pivot. To restore the pivot, we simply exchange it with the first element of G. Notice that the 4 is in its correct position in the sorted sequence and it is not considered any further.

Now the quicksort algorithm calls itself recursively, first to sort the sequence [image]; second to sort the sequence [image]. The quicksort of L selects 1 as the pivot, and creates the two subsequences [image] and [image]. Similarly, the quicksort of G uses 5 as the pivot and creates the two subsequences [image] and [image].

At this point in the example the recursion has been stopped. It turns out that to keep the code simple, quicksort algorithms usually stop the recursion when the length of a subsequence falls below a critical value called the cut-off. In this example, the cut-off is two (i.e., a subsequence of two or fewer elements is not sorted). This means that when the algorithm terminates, the sequence is not yet sorted. However as Figure gif shows, the sequence is almost sorted. In fact, every element is guaranteed to be less than two positions away from its final resting place.

We can complete the sorting of the sequence by using a straight insertion sort. In Section gif it is shown that straight insertion is quite good at sorting sequences that are almost sorted. In fact, if we know that every element of the sequence is at most d positions from its final resting place, the running time of straight insertion is O(dn) and since d=2 is a constant, the running time is O(n).

  • Implementation


Implementation

Program gif declares the abstract QuickSorter<T> class template. The QuickSorter class declares two versions of the DoSort routine, as well as the pure virtual function SelectPivot. Since SelectPivot is a pure virtual function, its implementation is given in a derived class.

program37764
Program: QuickSorter Class Definition

Program gif defines the DoSort routine of the QuickSorter<T> class that takes three arguments--a reference to the array to be sorted, array, and two integers, left and right, which denote left and right ends, respectively, of the sequence of the array to be sorted. I.e., this DoSort routine sorts

.

program37788
Program: QuickSorter<T> class Recursive DoSort Member Function Definition

As discussed above, the QuickSorter only sorts sequences whose length exceeds the cut-off value. Since the implementation shown only works correctly when the number of elements in the sequence to be sorted is three or more, the cut-off value of two is used (line 5).

The algorithm begins by calling the function SelectPivot which chooses one of the elements to be the pivot (line 7). The implementation of SelectPivot is discussed below. All that we require here is that the value p returned by SelectPivot satisfies [image]. Having selected an element to be the pivot, we hide the pivot by swapping it with the right-most element of the sequence (line 8). The pivot is hidden in order to get it out of the way of the next step.

The next step partitions the remaining elements into two sequences--one comprised of values less than or equal to the pivot, the other comprised of values greater than or equal to the pivot. The partitioning is done using two array indices, i and j. The first, i, starts at the left end and moves to the right; the second, j, starts at the right end and moves to the left.

The variable i is increased as long as array[i] is less than the pivot (line 14). Then the variable j is decreased as long as array[j] is greater than the pivot (line 15). When i and j meet, the partitioning is done (line 16). Otherwise, [image] but [image]. This situation is remedied by swapping array[i] and array[j] (line 17).

When the partitioning loop terminates, the pivot is still in array[right]; the value in array[i] is greater than or equal to the pivot; everything to the left is less than or equal to the pivot; and everything to the right is greater than or equal to the pivot. We can now put the pivot in its proper place by swapping it with array[i] (lines 19-20). This is called restoring the pivot. With the pivot in its final resting place, all we need to do is sort the subsequences on either side of the pivot (lines 21-24).

Program gif defines the main DoSort routine of the QuickSorter class. The main DoSort acts as the front end to the recursive DoSort given in Program gif . This routine takes as its lone argument a reference to the array to be sorted. It calls the recursive DoSort routine with left set to zero and right set to n-1, where n is the length of the array to be sorted. Finally, it uses a StraightInsertionSorter<T> to finish sorting the list.

program37838
Program: QuickSorter<T> Class Main DoSort Member Function Definition


Running Time Analysis

The running time of the recursive DoSort routine (Program gif ) is given by

where n is the number of elements in sequence to be sorted, [image] is the running time of the SelectPivot function, and i is the number of elements which end up to the left of the pivot, [image].

The running time of DoSort is affected by the SelectPivot routine in two ways: First, the value of the pivot chosen affects the sizes of the subsequences. I.e., the pivot determines the value i in Equation gif . Second, the running time of the SelectPivot routine itself, [image], must be taken into account. Fortunately, if [image], we can ignore its running time because there is already an O(n) term in the expression.

In order to solve Equation gif , we assume that [image] and then drop the [image] s from the recurrence to get

Clearly the solution depends on the value of i.

  • Worst-Case Running Time
  • Best-Case Running Time


Worst-Case Running Time

In the worst case the i in Equation gif is always zero. gif In this case, we solve the recurrence using repeated substitution like this:

The worst case occurs when the two subsequences are as unbalanced as they can be--one sequence has all the remaining elements and the other has none.


Best-Case Running Time

In the best case, the partitioning step divides the remaining elements into two sequences with exactly the same number of elements. For example, suppose that [image] for some integer m>0. After removing the pivot [image] elements remain. If these are divided evenly, each sequence will have [image] elements. In this case Equation gif gives


Average Running Time

To determine the average running time for the quicksort algorithm, we shall assume that each element of the sequence has an equal chance of being selected for the pivot. Therefore, if i is the number of elements in a sequence of length n less than the pivot, then i is uniformly distributed in the interval [0,n-1]. Consequently, the average value of [image]. Similarly, the average the value of [image]. To determine the average running time, we rewrite Equation gif thus:

To solve this recurrence we consider the case n>2 and then multiply Equation gif by n to get

Since this equation is valid for any n>2, by substituting n-1 for n we can also write

which is valid for n>3. Subtracting Equation gif from Equation gif gives

which can be rewritten as

Equation gif can be solved by telescoping like this:

Adding together Equation gif through Equation gif gives

where [image] is the [image] harmonic number . Finally, multiplying through by n+1 gives

In Section gif it is shown that [image], where [image] is called Euler's constant . Thus, we get that the average running time of quicksort is

Table gif summarizes the asymptotic running times for the quicksort routine and compares it to those of bubble sort. Notice that the best-case and average case running times for the quicksort algorithm have the same asymptotic bound!

Table: Running Times for Exchange Sorting
 | running time
algorithm | best case | average case | worst case
bubble sort |  |  |
quicksort (random pivot selection) |  |  |


Approach

One cannot learn to program just by reading a book. It is a skill that must be developed by practice. Nevertheless, the best practitioners study the works of others and incorporate their observations into their own practice. I firmly believe that after learning the rudiments of program writing, students should be exposed to examples of complex, yet well-designed program artifacts so that they can learn about the designing good software.

Consequently, this book presents the various data structures and algorithms as complete C++ program fragments. All the program fragments presented in this book have been extracted automatically from the source code files of working and tested programs.

The full functionality of the proposed draft ANSI standard C++ language is used in the examples--including templates, exceptions and run-time type information[3]. It has been my experience that by developing the proper abstractions, it is possible to present the concepts as fully functional programs without resorting to pseudo-code or to hand-waving.


Example-Geometric Series Summation Again

In this example we revisit the problem of computing a geometric series summation . We have already seen an algorithm to compute this summation in Section gif (Program gif ). This algorithm was shown to take [image] cycles.

The problem of computing the geometric series summation is identical to that of computing the value of a polynomial in which all of the coefficients are one. This suggests that we could make use of Horner's rule as discussed in Section gif . An algorithm to compute a geometric series summation using Horner's rule is given in Program gif .

program909
Program: Program to compute [image] using Horner's rule

The executable statements in Program gif comprise lines 3-6. Table gif gives the running times, as given by the simplified model, for each of these statements.

Table: Computing the running time of Program gif
statement | time
3 | 2
4a | 2
4b | 3(n+2)
4c | 4(n+1)
5 | 6(n+1)
6 | 2
TOTAL | 13n+22

In Programs gif and gif we have seen two different algorithms to compute the same geometric series summation. We determined the running time of the former to be [image] cycles and of the latter to be 13n+22 cycles. In particular, note that for all non-negative values of n, [image]. Hence, according to our simplified model of the computer, Program gif , which uses Horner's rule, always runs faster than Program gif !


Selecting the Pivot

The analysis in the preceding section shows that selecting a good pivot is important. If we do a bad job of choosing the pivot, the running time of quicksort is [image]. On the other hand, the average-case analysis shows that if every element of a sequence is equally likely to be chosen for the pivot, the running time is [image]. This suggests that we can expect to get good performance simply by selecting a random pivot!

If we expect to be sorting random input sequences, then we can achieve random pivot selection simply by always choosing, say, the first element of the sequence to be the pivot. Clearly this can be done in constant time. (Remember, the analysis requires that [image]). As long as each element in the sequence is equally likely to appear in the first position, the average running time will be [image].

In practice it is often the case that the sequence to be sorted is almost sorted. In particular, consider what happens if the sequence to be sorted using quicksort is already sorted. If we always choose the first element as the pivot, then we are guaranteed to have the worst-case running time! This is also true if we always pick the last element of the sequence. And it is also true if the sequence is initially sorted in reverse.

Therefore, we need to be more careful when choosing the pivot. Ideally, the pivot divides the input sequence exactly in two. I.e., the ideal pivot is the median element of the sequence. This suggests that the SelectPivot routine should find the median. To ensure that the running time analysis is valid, we need to find the median in O(n) time.

How do you find the median? One way is to sort the sequence and then select the [image] element. But this is not possible, because we need to find the median to sort the sequence in the first place!

While it is possible to find the median of a sequence of n elements in O(n) time, it is usually not necessary to do so. All that we really need to do is select a random element of the sequence while avoiding the problems described above.

A common way to do this is the median-of-three pivot selection technique. In this approach, we choose as the pivot the median of the element at the left end of the sequence, the element at the right end of the sequence, and the element in the middle of the sequence. Clearly, this does the right thing if the input sequence is initially sorted (either in forward or reverse order).

Program gif declares the MedianOfThreeQuickSorter<T> class template. The MedianOfThreeQuickSorter<T> class is derived from QuickSorter<T> abstract base class. It provides an implementation for the SelectPivot function based on median-of-three pivot selection. Notice that this algorithm does exactly three comparisons to select the pivot. As a result, its running time is O(1). In practice this scheme performs sufficiently well that more complicated pivot selection approaches are unnecessary.

program38053
Program: MedianOfThreeQuickSorter<T> ClassSelectPivot Member Function Definition


Selection Sorting

The third class of sorting algorithm that we consider comprises algorithms that sort by selection . Such algorithms construct the sorted sequence one element at a time by adding elements to the sorted sequence in order. At each step, the next element to be added to the sorted sequence is selected from the remaining elements.

Because the elements are added to the sorted sequence in order, they are always added at one end. This is what makes selection sorting different from insertion sorting. In insertion sorting elements are added to the sorted sequence in an arbitrary order. Therefore, the position in the sorted sequence at which each subsequent element is inserted is arbitrary.

Both selection sorts described in this section sort the arrays in place . Consequently, the sorts are implemented by exchanging array elements. Nevertheless, selection differs from exchange sorting because at each step we select the next element of the sorted sequence from the remaining elements and then we move it into its final position in the array by exchanging it with whatever happens to be occupying that position.

  • Straight Selection Sorting
  • Sorting with a Heap
  • Building the Heap


Straight Selection Sorting

The simplest of the selection sorts is called straight selection . Figure gif illustrates how straight selection works. In the version shown, the sorted list is constructed from the right (i.e., from the largest to the smallest element values).

figure38078
Figure: Straight Selection Sorting

At each step of the algorithm, a linear search of the unsorted elements is made in order to determine the position of the largest remaining element. That element is then moved into the correct position of the array by swapping it with the element which currently occupies that position.

For example, in the first step shown in Figure gif , a linear search of the entire array reveals that 9 is the largest element. Since 9 is the largest element, it belongs in the last array position. To move it there, we swap it with the 4 that initially occupies that position. The second step of the algorithm identifies 6 as the largest remaining element an moves it next to the 9. Each subsequent step of the algorithm moves one element into its final position. Therefore, the algorithm is done after n-1 such steps.

  • Implementation


Implementation

Program gif defines the StraightSelectionSorter<T> class. This class is derived from the abstract Sorter<T> base class and it provides an implementation for DoSort. The DoSort routine follows directly from the algorithm discussed above. In each iteration of the main loop (lines 11-18), exactly one element is selected from the unsorted elements and moved into the correct position. A linear search of the unsorted elements is done in order to determine the position of the largest remaining element (lines 14-16). That element is then moved into the correct position (line 17).

program39246
Program: StraightSelectionSorter<T> Class DoSort Member Function Definition

In all n-1 iterations of the outer loop are needed to sort the array. Notice that exactly one swap is done in each iteration of the outer loop. Therefore, n-1 data exchanges are needed to sort the list.

Furthermore, in the [image] iteration of the outer loop, i-1 iterations of the inner loop are required and each iteration of the inner loop does one data comparison. Therefore, [image] data comparisons are needed to sort the list.

The total running time of the straight selection DoSort routine is [image]. Because the same number of comparisons and swaps are always done, this running time bound applies in all cases. I.e., the best-case, average-case and worst-case running times are all [image].


Sorting with a Heap

Selection sorting involves the repeated selection of the next element in the sorted sequence from the set of remaining elements. For example, the straight insertion sorting algorithm given in the preceding section builds the sorted sequence by repeatedly selecting the largest remaining element and prepending it to the sorted sequence developing at the right end of the array.

At each step the largest remaining element is withdrawn from the set of remaining elements. A linear search is done because the order of the remaining elements is arbitrary. However, if we consider the value of each element as its priority, we can view the set of remaining elements as a priority queue. In effect, a selection sort repeatedly dequeues the highest priority element from a priority queue.

Chapter gif presents a number of priority queue implementations, including binary heaps, leftist heaps and binomial queues. In this section we present a version of selection sorting that uses a binary heap to hold the elements that remain to be sorted. Therefore, it is called a heapsort . The principal advantage of using a binary heap is that it is easily implemented using an array and the entire sort can be be done in place.

As explained in Section gif , a binary heap is a complete binary tree which is easily represented in an array. The n nodes of the heap occupy positions 1 through n of the array. The root is at position 1. In general, the children of the node at position i of the array are found at positions 2i and 2i+1, and the parent is found at position [image].

The heapsort algorithm consists of two phases. In the first phase, the unsorted array is transformed into a heap. (This is called heapifying the array). In this case, a max-heap rather than a min-heap is used. The data in a max heap satisfies the following condition: For every node in the heap that has a parent, the item contained in the parent is greater than or equal to the item contained in the given node.

The second phase of heapsort builds the sorted list. The sorted list is built by repeatedly selecting the largest element, withdrawing it from the heap, and adding it to the sorted sequence. As each element is withdrawn from the heap, the remaining elements are heapified.

  • Implementation


Implementation

Program gif defines the class template HeapSorter<T>. The HeapSorter<T> class is derived from the abstract Sorter<T> base class. Three protected member functions are defined--BuildHeap, PercolateDown and DoSort. The DoSort routine comprises the body of the sorting algorithm. The BuildHeap and PercolateDown routines are used by DoSort to build the heap and then to sort the array.

program39283
Program: HeapSorter<T> Class Definition

In the first phase of heapsort, the unsorted array is transformed into a max heap. Throughout the process we view the array as a complete binary tree. Since the data in the array is initially unsorted, the tree is not initially heap-ordered. We make the tree into a max heap from the bottom up. I.e., we start with the leaves and work towards the root. Figure gif illustrates this process.

figure39297
Figure: Combining Heaps by Percolating Values

Figure gif (a) shows a complete tree that is not yet heap ordered--the root is smaller than both its children. However, the two subtrees of the root are heap ordered. Given that both of the subtrees of the root are already heap ordered, we can heapify the tree by percolating the value in the root down the tree.

To percolate a value down the tree, we swap it with its largest child. E.g., in Figure gif (b) we swap 3 and 7. Swapping with the largest child ensures that after the swap, the new root is greater than or equal to both its children.

Notice that after the swap the heap-order is satisfied at the root, but not in the left subtree of the root. We continue percolating the 3 down by swapping it with 6 as shown in Figure gif (c). In general, we percolate a value down either until it arrives in a position in which the heap order is satisfied or until it arrives in a leaf. As shown in Figure gif (d), the tree obtained when the percolation is finished is a max heap

The PercolateDown routine shown in Program gif implements the algorithm described above. The PercolateDown routine takes three arguments: a reference to the array; the number of elements in the array to be considered, n; and the position, i, of the node to be percolated.

program40113
Program: HeapSorter<T> Class PercolateDown Member Function Definition

The purpose of the PercolateDown routine is to transform the subtree rooted at position i into a max heap. It is assumed that the left and right subtrees of the node at position i are already max heaps. Recall that the children of node i are found at positions 2i and 2i+1. PercolateDown percolates the value in position i down the tree by swapping elements until the value arrives in a leaf node or until both children of i contain smaller value.

A constant amount of work is done in each iteration. Therefore, the running time of the PercolateDown routine is determined by the number of iterations of its main loop (lines 5-15). In fact, the number of iterations required in the worst case is equal to the height in the tree of node i.

Since the root of the tree has the greatest height, the worst-case occurs for i=1. In Chapter gif it is shown that the height of a complete binary tree is [image]. Therefore the worst-case running time of the PercolateDown routine is [image].

Recall that BuildHeap calls PercolateDown for [image]. If we assume that the worst-case occurs every time, the running time of BuildHeap is [image].


Building the Heap

The BuildHeap routine shown in Program gif transforms an unsorted array into a max heap. It does so by calling the PercolateDown routine for [image].

program40134
Program: HeapSorter<T> Class BuildHeap Member Function Definition

Why does BuildHeap start percolating at [image]? A complete binary tree with n nodes has exactly [image] leaves. Therefore, the last node in the array which has a child is in position [image]. Consequently, the BuildHeap routine starts doing percolate down operations from that point.

The BuildHeap visits the array elements in reverse order. In effect the algorithm starts at the deepest node that has a child and works toward the root of the tree. Each array position visited is the root of a subtree. As each such subtree is visited, it is transformed into a max heap. Figure gif illustrates how the BuildHeap routine heapifies an array that is initially unsorted.

figure40148
Figure: Building A Heap

  • Running Time Analysis
  • The Sorting Phase


Running Time Analysis

The BuildHeap routine does exactly [image] PercolateDown operations. As discussed above, the running time for PercolateDown is [image], where [image] is the height in the tree of the node at array position i. The highest node in the tree is the root and its height is [image]. If we make the simplifying assumption that the running time for PercolateDown is [image] for every value of i, we get that the total running time for BuildHeap is [image].

However, [image] is not a tight bound. The maximum number of iterations of the PercolateDown loop done during the entire process of building the heap is equal to the sum of the heights of all of the nodes in the tree! The following theorem shows that this is O(n).

Theorem Consider a perfect binary tree T of height h having [image] nodes. The sum of the heights of the nodes in T is [image].

extbfProof A perfect binary tree has 1 node at height h, 2 nodes at height h-1, 4 nodes at height h-2 and so on. In general, there are [image] nodes at height h-i. Therefore, the sum of the heights of the nodes is [image].

The summation can be solved as follows: First, we make the simple variable substitution i=j-1:

Note that the summation which appears on the right hand side is identical to that on the left. Rearranging Equation gif and simplifying gives:

It follows directly from Theorem gif that the sum of the heights of a perfect binary tree is O(n). But a heap is not a perfect tree--it is a complete tree. Nevertheless, it is easy to show that the same bound applies to a complete tree. The proof is left as an exercise for the reader (Exercise gif ). Therefore, the running time for the BuildHeap routine is O(n), were n is the length of the array to be heapified.


The Sorting Phase

Once the max heap has been built, heapsort proceeds to the selection sorting phase. In this phase the sorted sequence is obtained by repeatedly withdrawing the largest element from the max heap. Figure gif illustrates how this is done.

figure41101
Figure: Heap Sorting

The largest element of the heap is always found at the root and the root of a complete tree is always in array position one. Suppose the heap occupies array positions 1 through k. When an element is withdrawn from the heap, its length decreases by one. I.e., after the withdrawal the heap occupies array positions 1 through k-1. Thus, array position k is no longer required by the max heap. However, the next element of the sorted sequence belongs in position k!

So, the sorting phase of heapsort works like this: We repeatedly swap the largest element in the heap (always in position 1) into the next position of the sorted sequence. After each such swap, there is a new value at the root of the heap and this new value is pushed down into the correct position in the heap using the PercolateDown routine.

Program gif gives the DoSort routine of the HeapSorter<T> class. The DoSort routine embodies both phases of the heapsort algorithm. However, before it starts the first phase of the algorithm, DoSort sets the array base to one. This modification of the array base simplifies the coding of the algorithms. As discussed in Section gif the array base will have been set to zero by the Sort member function of the Sorter<T> base class and will be restored to the original base by that routine.

program42824
Program: HeapSorter<T> Class DoSort Member Function Definition

In the first phase of heapsort the BuildHeap routine is called to transform the array into a max heap. As discussed above, this is done in O(n) time.

The second phase of the heapsort algorithm builds the sorted list. In all n-1 iterations of the loop on lines 6-10 are required. Each iteration involves one swap followed by a PercolateDown operation. Since the worst-case running time for PercolateDown is [image], the total running time of the loop is [image]. The running time of the second phase asymptotically dominates that of the first phase. As a result, the worst-case running time of heapsort is [image].


Merge Sorting

The fourth class of sorting algorithm we consider comprises algorithms that sort by merging . Merging is the combination of two or more sorted sequences into a single sorted sequence.

Figure gif illustrates the basic, two-way merge operation. In a two-way merge, two sorted sequences are merged into one. Clearly, two sorted sequences each of length n can be merged into a sorted sequence of length 2n in O(2n)=O(n) steps. However in order to do this, we need space in which to store the result. I.e., it is not possible to merge the two sequences in place in O(n) steps.

figure42842
Figure: Two-Way Merging

Sorting by merging is a recursive, divide-and-conquer strategy. In the base case, we have a sequence with exactly one element in it. Since such a sequence is already sorted, there is nothing to be done. To sort a sequence of n>1 elements:

  1. Divide the sequence into two sequences of length [image] and [image];
  2. recursively sort each of the two subsequences; and then,
  3. merge the sorted subsequences to obtain the final result.
Figure gif illustrates the operation of the two-way merge sort algorithm.

figure44016
Figure: Two-Way Merge Sorting

  • Implementation
  • Merging
  • Two-Way Merge Sorting
  • Running Time Analysis


About Geometric Series Summation

The series, [image], is a geometric series and the summation

is called the geometric series summation .

The summation can be solved as follows: First, we make the simple variable substitution i=j-1:

Note that the summation which appears on the right is identical to the left hand side. Rearranging Equation gif , and simplifying gives


Implementation

Program gif declares the TwoWayMergeSorter<T> class template. A single member variable, tempArray, is declared. This variable is a pointer to an Array<T> instance. Since merge operations cannot be done in place, a second, temporary array is needed. The tempArray variable is keeps track of that array. The TwoWayMergeSorter constructor simply sets the tempArray pointer to zero.

program44548
Program: TwoWayMergeSorter<T> Class Definition

In addition to the constructor, three protected member functions are declared, Merge and two versions of DoSort. The purpose of the Merge function is to merge sorted subsequences of the array to be sorted. The two DoSort routines implement the sorting algorithm itself.


Merging

The Merge function of the TwoWayMergeSorter<T> class is defined in Program gif . Altogether, this function takes four parameters: The first is a reference to the array to be sorted. The remaining three, left, middle, and right, are unsigned integers. It is assumed that

Furthermore, it is assumed that the two subsequences of the array,

and

are both sorted. The Merge routine merges the two sorted subsequences using the temporary array specified by tempArray. It then copies the merged (and sorted) sequence into the array at

program44595
Program: TwoWayMergeSorter<T> Class Merge Member Function Definition

In order to determine the running time of the Merge routine it is necessary to recognize that the total number of iterations of the three loops (lines 8-14, lines 15-16, and lines 17-18) is [image]. The total number of iterations of the fourth loop (lines 19-20) is the same. Since all the loop bodies do a constant amount of work, the total running time for the Merge routine is O(n), where [image] is the total number of elements in the two subsequences that are merged.


Two-Way Merge Sorting

Program gif gives the code for the two DoSort member functions of the TwoWayMergeSorter class. The main DoSort routine takes as its lone argument a reference to the array to be sorted (line 2). First, it allocates a temporary array, the length of which is equal to the length of the array to be sorted (line 4). Then it calls the second, recursive DoSort routine which sorts the array (line 5). After the array has been sorted, the main DoSort deletes the temporary array (line 6).

program44617
Program: TwoWayMergeSorter<T> Class DoSort Member Function Definitions

The second DoSort routine implements the recursive, divide-and-conquer merge sort algorithm described above. The routine takes three parameters--array, left and right. The first is a reference to the array to be sorted and the latter two specify the subsequence of the array to be sorted. If the sequence to be sorted contains more than one element, the sequence is split in two (line 16), each half is recursively sorted (lines 17-18), and then two sorted halves are merged (line 19).


Running Time Analysis

The running time of merge sort is determined by the running time of the recursive DoSort routine. (The main DoSort only adds a constant amount of overhead). The running time of the recursive DoSort routine is given by the following recurrence:

where [image].

In order to simplify the solution of Equation gif we shall assume that [image] for some integer [image]. Dropping the [image] s from the equation we get

which is easily solved by repeated substitution:

Therefore, the running time of merge sort is [image].


A Lower Bound on Sorting

The preceding sections present three [image] sorting algorithms--quicksort, heapsort, and the two-way merge sort. But is [image] the best we can do? In this section we answer the question by showing that any sorting algorithm that sorts using only binary comparisons must make [image] such comparisons. If each binary comparison takes a constant amount of time, then running time for any such sorting algorithm is also [image].

Consider the problem of sorting the sequence [image] comprised of three distinct items. I.e., [image]. Figure gif illustrates a possible sorting algorithm in the form of a decision tree . Each node of the decision tree represents one binary comparison. I.e., in each node of the tree, exactly two elements of the sequence are compared. Since there are exactly two possible outcomes for each comparison, each non-leaf node of the binary tree has degree two.

figure44650
Figure: A Decision Tree for Comparison Sorting

For example, suppose that a<b<c. Consider how the algorithm shown in Figure gif discovers this. The first comparison compares a and b which reveals that a<b. The second comparison compares a and c to find that a<c. At this point it has been determined that a<b and a<c-- the relative order of b and c is not yet known. Therefore, one more comparison is required to determine that b<c. Notice that the algorithm shown in Figure gif works correctly in all cases because every possible permutation of the sequence S appears as a leaf node in the decision tree. Furthermore, the number of comparisons required in the worst case is equal to the height of the decision tree!

Any sorting algorithm that uses only binary comparisons can be represented by a binary decision tree. Furthermore, it is the height of the binary decision tree that determines the worst-case running time of the algorithm. In general, the size and shape of the decision tree is a function of the sorting algorithm and the number of items to be sorted.

Given an input sequence of n items to be sorted, every binary decision tree that correctly sorts the input sequence must have at least n! leaves--one for each permutation of the input. Therefore, it follows directly from Theorem gif that the height of the binary decision tree is at least [image]:

Since the height of the decision tree is [image], the number of comparisons done by any sorting algorithm that sorts using only binary comparisons is [image]. Assuming each comparison can be done in constant time, the running time of any such sorting algorithm is [image].


Distribution Sorting

The final class of sorting algorithm considered in this chapter consists of algorithms that sort by distribution . The unique characteristic of a distribution sorting algorithm is that it does not make use of comparisons to do the sorting.

Instead, distribution sorting algorithms rely on a priori knowledge about the universal set from which the elements to be sorted are drawn. For example, if we know a priori that the size of the universe is a small, fixed constant, say m, then we can use the bucket sorting algorithm described in Section gif .

Similarly, if we have a universe the elements of which can be represented with a small, finite number of bits (or even digits, letters, or symbols), then we can use the radix sorting algorithm given in Section gif .

  • Bucket Sort
  • Radix Sort


Bucket Sort

Bucket sort is possibly the simplest distribution sorting algorithm. The essential requirement is that the size of the universe from which the elements to be sorted are drawn is a small, fixed constant, say m.

For example, suppose that we are sorting elements drawn from [image], i.e., the set of integers in the interval [0,m-1]. Bucket sort uses m counters. The [image] counter keeps track of the number of occurrences of the [image] element of the universe. Figure gif illustrates how this is done.

figure45218
Figure: Bucket Sorting

In Figure gif , the universal set is assumed to be [image]. Therefore, ten counters are required--one to keep track of the number of zeroes, one to keep track of the number of ones, and so on. A single pass through the data suffices to count all of the elements. Once the counts have been determined, the sorted sequence is easily obtained. E.g., the sorted sequence contains no zeroes, two ones, one two, and so on.

  • Implementation


Implementation

Program gif gives the declaration of the BucketSorter class. Notice that the BucketSorter class is not a template. This bucker sorter is designed to sort specifically an array of unsigned ints. The BucketSorter class contains two member variables, m and count. The unsigned integer m simply keeps track of the size of the universe. The count variable is an array of unsigned integers used to count the number of occurrences of each element of the universal set.

program45590
Program: BucketSorter Class Definition

The constructor for the BucketSorter class takes a single argument which specifies the size of the universal set. The variable m is set to the specified value, and the count array is initialized to have the required size.

The DoSort routine of the BucketSorter is defined in Program gif . This routine is passed a reference to the array of data to be sorted. DoSort begins by setting all of the counters to zero (lines 3-4). This can clearly be done in O(m) time.

program45608
Program: BucketSorter Class DoSort Member Function Definition

Next, a single pass is made through the data to count the number of occurrences of each element of the universe (lines 5-6). Since each element of the array is examined exactly once, the running time is O(n).

In the final step, the sorted output sequence is created (lines 7-9). Since the output sequence contains exactly n items, the body of the inner loop (line 9) is executed exactly n times. During the [image] iteration of the outer loop (line 7), the loop termination test of the inner loop (line 8) is evaluated [image] times. As a result, the total running time of the final step is O(m+n).

Thus, the running time of the bucket sort routine is O(m+n). Note that if m=O(n), the running time for bucket sort is O(n). I.e., the bucket sort algorithm is a linear-time sorting algorithm! Bucket sort breaks the [image] bound associated with sorting algorithms that use binary comparisons because bucket sort does not do any binary comparisons. The cost associated with breaking the [image] running time bound is the O(m) space required for the array of counters. Consequently, bucket sort is practical only for small m. E.g., to sort 16-bit integers using bucket sort requires the use of an array of [image] counters.


Radix Sort

This section presents a sorting algorithm known as least-significant-digit-first radix sorting . Radix sorting is based on the bucket sorting algorithm discussed in the preceding section. However, radix sorting is practical for much larger universal sets than it is practical to handle with a bucket sort.

Radix sorting can be used when each element of the universal set can be viewed as a sequences of digits (or letters or any other symbols). For example, we can represent each integer between 0 and 99 as a sequence of two, decimal digits. (E.g., the number five is represented as ``05'').

To sort an array of two-digit numbers, the algorithm makes two sorting passes through the array. In the first pass, the elements of the array are sorted by the least significant decimal digit. In the second pass, the elements of the array are sorted by the most significant decimal digit. The key characteristic of the radix sort is that the second pass is done in such a way that it does not destroy the effect of the first pass. Consequently, after two passes through the array, the data is contained therein is sorted.

Each pass of the radix sort is implemented as a bucket sort. In the example we base the sort on decimal digits. Therefore, this is called a radix-10 sort and ten buckets are required to do each sorting pass.

Figure gif illustrates the operation of the radix-10 sort. The first radix sorting pass considers the least significant digits. As in the bucket sort, a single pass is made through the unsorted data, counting the number of times each decimal digit appears as the least-significant digit. E.g., there are no elements that have a 0 as the least-significant digit; there are two elements that have a 1 as the least-significant digit; and so on.

figure45631
Figure: Radix Sorting

After the counts have been determined, it is necessary to permute the input sequence so that it is sorted by the least-significant digits. To do this permutation efficiently, we compute the sequence of offsets given by

where R is the sorting radix. Note that [image] is the position in the permuted sequence of the first occurrence of an element whose least significant digit is i. By making use of the offsets, it is possible to permute the input sequence by making a single pass through the sequence.

The second radix sorting pass considers the most significant digits. As above a single pass is made through the permuted data sequence counting the number of times each decimal digit appears as the most-significant digit. Then the sequence of offsets is computed as above. The sequence is permuted again using the offsets producing the final, sorted sequence.

In general, radix sorting can be used when the elements of the universe can be viewed as p-digit numbers with respect to some radix, R. I.e., each element of the universe has the form

where [image] for [image]. In this case, the radix sort algorithm must make p sorting passes from the least significant digit, [image], to the most significant digit, [image], and each sorting pass uses exactly R counters.

Radix sorting can also be used when the universe can be viewed as the cross-product of a finite number of finite sets. I.e., when the universe has the form

where p>0 is a fixed integer constant and [image] is a finite set for [image]. For example, each card in a 52-card deck of playing cards can be represented as an element of [image], where [image] and [image].

Before we can sort over the universe U, we need to define what it means for one element to precede another in U. The usual way to do this is called lexicographic ordering . For example in the case of the playing cards we may say that one card precedes another if its suit precedes the other suit or if the suits are equal but the face value precedes that of the other.

In general, given the universe [image], and two elements of U, say x and y, represented by the p-tuples [image] and [image], respectively, we say that x lexicographically precedes y if there exists [image] such that [image] and [image] for all [image].

With this definition of precedence, we can radix sort a sequence of elements drawn from U by sorting with respect to the components of the p-tuples. Specifically, we sort first with respect to [image], then [image], and so on down to [image]. Notice that the algorithm does p sorting passes and in the [image] pass it requires [image] counters. For example to sort a deck of cards, two passes are required. In first pass the cards are sorted into 13 piles according to their face values. In the second pass the cards are sorted into four piles according to their suits.

  • Implementation


Implementation

Program gif gives the declaration of the RadixSorter class. Notice that the RadixSorter class is not a template. This radix sorter is designed to sort specifically an array of unsigned ints.

Three constants are declared as static members of the in the RadixSorter class--R, r and p. The constant R represents the radix and [image]. The constant p is the number sorting passes needed to sort the data.

The RadixSorter class contains one member variable--count. The count variable is an array of unsigned integers used to implement the sorting passes. The RadixSorter constructor simply initializes the count array with length R.

program46493
Program: RadixSorter Class Definition

Program gif defines the constants R, r and p, and gives the code for the DoSort member function of the RadixSorter class. In this case r=8 and [image]. Therefore, a radix-256 sort is being done. We have chosen R as a power of two because that way the computations required to implement the radix sort can be implemented efficiently using simple bit shift and mask operations. In order to sort b-bit unsigned integers, it is necessary to make [image] sorting passes. The constants r, R and p are initialized accordingly.

program46513
Program: RadixSorter Class DoSort Member Function Definition

DoSort begins by creating a temporary array of unsigned ints of length n. Each iteration of the main loop corresponds to one pass of the radix sort (lines 9-30). In all p iterations are required.

During the [image] pass of the main loop the following steps are done: First, the R counters are all set to zero (lines 11-12). This takes O(R) time. Then a pass is made through the input array during which the number of occurrences of each radix-R digit in the [image] digit position are counted (lines 13-17). This pass takes O(n) time. Notice that during this pass all the input data is copied into the temporary array.

Next, the array of counts is transformed into an array of offsets according to Equation gif . This requires a single pass through the counter array (lines 18-24). Therefore, it takes O(R) time. Finally, the data sequence is permuted by copying the values from the temporary array back into the input array (lines 25-30). Since this requires a single pass through the data arrays, the running time is O(n).

After the p sorting passes have been done, the array of data is sorted. The running time for the DoSort routine of the RadixSorter class is [image]. If we assume that the size of an integer is 32 bits and given that R=256, the number of sorting passes required is p=4. Therefore, the running time for the radix sort is simply O(n). I.e., radix sort is a linear-time sorting algorithm.


Example-Computing Powers

In this section we consider the running time to raise a number to a given integer power. I.e., given a value x and non-negative integer n, we wish to compute the [image]. A naıve way to calculate [image] would be to use a loop such as

int result = 1;
for (unsigned int i = 0; i <= n; ++i)
    result *= x;
While this may be fine for small values of n, for large values of n the running time may become prohibitive. As an alternative, consider the following recursive definition

For example, using Equation gif , we would determine [image] as follows

which requires a total of five multiplication operations. Similarly, we would compute [image] as follows

which requires a total of eight multiplication operations.

A recursive algorithm to compute [image] based on the direct implementation of Equation gif is given in Program gif . Table gif gives the running time, as predicted by the simplified model, for each of the executable statements in Program gif .

program993
Program: Program to compute

Table: Computing the running time of Program gif
 | time
statement | n=0 | n>0 | n>0
 |  | n is even | n is odd
3 | 3 | 3 | 3
4 | 2 | -- | --
5 | -- | 5 | 5
6 | -- |  | --
8 | -- | -- |
TOTAL | 5 |  |

By summing the columns in Table gif we get the following recurrence for the running time of Program gif

As the first attempt at solving this recurrence, let us suppose that [image] for some k>0. Clearly, since n is a power of two, it is even. Therefore, [image].

For [image], Equation gif gives

This can be solved by repeated substitution:

The substitution stops when k=j. Thus,

Note that if [image], then [image]. In this case, running time of Program gif is [image].

The preceding result is, in fact, the best case--in all but the last two recursive calls of the function, n was even. Interestingly enough, there is a corresponding worst-case scenario. Suppose [image] for some value of k>0. Clearly n is odd, since it is one less than [image] which is a power of two and even. Now consider [image]:

Hence, [image] is also odd!

For example, suppose n is 31 ( [image]). To compute [image], Program gif calls itself recursively to compute [image], [image], [image], [image], and finally, [image] --all but the last of which are odd powers of x.

For [image], Equation gif gives

Solving this recurrence by repeated substitution we get

The substitution stops when k=j. Thus,

Note that if [image], then [image]. In this case, running time of Program gif is [image].

Consider now what happens for an arbitrary value of n. Table gif shows the recursive calls made by Program gif in computing [image] for various values of n.

Table: Recursive calls made in Program gif
n |  | powers computed recursively
1 | 1 |
2 | 2 |
3 | 2 |
4 | 3 |
5 | 3 |
6 | 3 |
7 | 3 |
8 | 4 |

By inspection we determine that the number of recursive calls made in which the second argument is non-zero is [image]. Furthermore, depending on whether the argument is odd or even, each of these calls contributes either 18 or 20 cycles. The pattern emerging in Table gif suggests that, on average just as many of the recursive calls result in an even number as result in an odd one. The final call (zero argument) adds another 5 cycles. So, on average, we can expect the running time of Program gif to be


Performance Data

In order to better understand the actual performance of the various sorting algorithms presented in this chapter, it is necessary to conduct some experiments. Only by conducting experiments is it possible to determine the relative performance of algorithms with the same asymptotic running time.

To measure the performance of a sorting algorithm, we need to provide it with some data to sort. To obtain the results presented here, random sequences of unsigned integers were sorted. I.e., for each value of n, the RandomNumberGenerator class defined in Section gif was used to create a sequence of n integers. In all cases (except for bucket sort) the random numbers are uniformly distributed in the interval [image]. For the bucket sort the numbers are uniformly distributed in [image].

Figures gif , gif and gif show the actual running times of the sorting algorithms presented in this chapter. These running times were measured on a Sun SPARCstation 5, Model 85, which has an 85 MHz clock, and 32MB RAM. The programs were compiled using the SPARCompiler C++ 4.1 compiler, and run under the Solaris 2.3 operating system. The times shown are user CPU times, measured in seconds.

Figure gif shows the running times of the [image] sorts for sequences of length n, [image]. Notice that the bubble sort has the worst performance and that the binary insertion sort has the best performance. Figure gif clearly shows that, as predicted, binary insertion is better than straight insertion. Notice too that all of the [image] sorts require in excess of two minutes of execution time to sort an array of 20000 integers.

figure46539
Figure: Actual Running Times of the [image] Sorts

The performance of the [image] sorts is shown in Figure gif . In this case, the length of the sequence varies between n=100 and [image]. The graph clearly shows that the [image] algorithms are significantly faster that the [image] ones. All three algorithms sort 100000 integers in under 6 seconds. Clearly, quick sort is the best of the three, whereas the two-way merge sort and heap sort have similar running times.

figure47234
Figure: Actual Running Times of the [image] Sorts

Figure gif shows the actual running times for the bucket sort and radix sort algorithms. Both these algorithms were shown to be O(n) sorts. The graph shows results for n between 100 and [image]. The universe used to test bucket sort was [image]. I.e., a total of m=1024 counters (buckets) were used. For the radix sort, 32-bit integers were sorted by using the radix R=256 and doing p=4 sorting passes.

figure47470
Figure: Actual Running Times of the O(n) Sorts

Clearly, the bucket sort has the best running time. For example, it sorts one million 10-bit integers in about 5 seconds. Radix sort performs extremely well too. It sorts one-million 32-bit integers in about 20 seconds. This is a factor of four slower than the bucket sort due largely to the fact that radix sort does four sorting passes whereas bucket sort requires only one.


Exercises

  1. Consider the sequence of integers

    For each of the following sorting algorithms, draw a sequence of diagrams that traces the execution of the algorithm as it sorts the sequence S: straight insertion sort, binary insertion sort, bubble sort, quick sort, straight selection sort, heapsort, merge sort, and bucket sort.

  2. Draw a sequence of diagrams that traces the execution of a radix-10 sort of the sequence

  3. For each of the sorting algorithms listed in Exercises gif and gif indicate whether the sorting algorithm is stable.
  4. Consider a sequence of three distinct keys [image]. Draw the binary decision tree that represents each of the following sorting algorithms: straight insertion sort, straight selection sort, and bubble sort.
  5. Devise an algorithm to sort a sequence of exactly three elements. Make your algorithm as efficient as possible.
  6. Prove that the swapping of a pair of adjacent elements removes at most one inversion from a sequence.
  7. Consider the sequence of elements [image]. What is the maximum number of inversions that can be removed by the swapping of a pair of distinct elements [image] and [image]? Express the result in terms of the distance between [image] and [image]: .
  8. Devise a sequence of keys such that exactly eleven inversions are removed by the swapping of one pair of elements.
  9. Prove that binary insertion sort requires [image] comparisons.
  10. Consider an arbitrary sequence [image]. To sort the sequence, we determine the permutation [image] such that

    Prove that bubble sort requires at least p passes where

  11. Modify the bubble sort algorithm (Program gif ) so that it terminates the outer loop when it detects that the array is sorted. What is the running time of the modified algorithm? Hint: See Exercise gif .
  12. A variant of the bubble sorting algorithm is the so-called odd-even transposition sort . Like bubble sort, this algorithm a total of n-1 passes through the array. Each pass consists of two phases: The first phase compares [image] with [image] and swaps them if necessary for all the odd values of of i. The second phase does the same for the even values of i.
    1. Show that the array is guaranteed to be sorted after n-1 passes.
    2. What is the running time of this algorithm?
  13. Another variant of the bubble sorting algorithm is the so-called cocktail shaker sort . Like bubble sort, this algorithm a total of n-1 passes through the array. However, alternating passes go in opposite directions. E.g., during the first pass the largest item bubbles to the end of the array and during the second pass the smallest item bubbles to the beginning of the array.
    • Show that the array is guaranteed to be sorted after n-1 passes.
    • What is the running time of this algorithm?
  14. Consider the following algorithm for selecting the [image] largest element from an unsorted sequence of of n elements, [image].
    1. If [image], sort S and select directly the [image] largest element.
    2. Otherwise n>5: Partition the sequence S into subsequences of length five. In general, there will be [image] subsequences of length five and one of length [image].
    3. Sort by any means each of the subsequences of length five. (See Exercise gif ).
    4. Form the sequence [image] containing the [image] median values of each of the subsequences of length five.
    5. Apply the selection procedure recursively to find the median element of M. Let m be the median of the medians.
    6. Partition S into three subsequences, [image]. such that all the elements in L are less than m, all the elements in E are equal to m, and all the elements of G are greater than m.
    7. If [image] then apply the procedure recursively to select the [image] largest element of L; if [image], the result is m; otherwise apply the procedure recursively to select the [image] largest element of G.
    1. What is the running time of this algorithm?
    2. Show that if we use this algorithm to select the pivot the worst-case running time of quick sort is [image].
  15. Show that the sum of the heights of the nodes in a complete binary tree with n nodes altogether is n-b(n), where b(n) is the number of ones in the binary representation of n.


Projects

  1. Design and implement an algorithm that finds all the duplicates in a random sequence of keys.
  2. Suppose that instead of an Array<T> instance, we wish to sort a sequence of data represented using the linked-list class LinkedList<T>. Which of the sorting algorithms described in this chapter is the most appropriate for sorting a linked list? Design and implement a linked list sorter class that implements this algorithm.
  3. Replace the DoSort routine of the MergeSorter class with a non-recursive version. What is the running time of the non-recursive merge sort?
  4. Replace the DoSort routine of the QuickSorter class with a non-recursive version. What is the running time of the non-recursive quick sort? Hint: Use a stack.
  5. Design and implement a radix-sorter class that sorts an array of string class instances.
  6. Design and implement a RandomPivotQuickSorter class that uses a random number generator (see Section gif ) to select a pseudorandom pivot. Run a sequence of experiments to compare the running times of random pivot selection with median-of-three pivot selection.
  7. Design and implement a MeanPivotQuickSorter class that partitions the sequence to be sorted into elements that are less than the mean and elements that are greater than the mean. Run a sequence of experiments to compare the running times of the mean pivot quick sorter with median-of-three pivot selection.
  8. Design and implement a MedianPivotQuickSorter class that uses the algorithm given in Exercise gif to select the median element for the pivot. Run a sequence of experiments to compare the running times of median pivot selection with median-of-three pivot selection.
  9. Design and implement a sorter class that sorts using a PriorityQueue instance. (See Chapter gif ).


Graphs and Graph Algorithms

A graph is simply a set of points together with a set of lines connecting various points. Myriad real-world application problems can be reduced to problems on graphs.

Suppose you are planning a trip by airplane. From a map you have determined the distances between the airports in the various cities that you wish to visit. The information you have gathered can be represented using a graph as shown in Figure gif (a). The points in the graph represent the cities and the lines represent the distances between them. Given such a graph, you can answer questions such as ``What is the shortest distance between LAX and JFK?'' or ``What is the shortest route that visits all of the cities?''

figure47927
Figure: Real-World Examples of Graphs

An electric circuit can also be viewed as a graph as shown in Figure gif (b). In this case the points in the graph indicate where the components are connected (i.e., the wires) and the lines represent the components themselves (e.g, resistors and capacitors). Given such a graph, we can answer questions such as ``What are the mesh equations that describe the circuit's behavior?''

Similarly, a logic circuit can be reduced to a graph as shown in Figure gif (c). In this case the logic gates are represented by the points and arrows represent the signal flows from gate outputs to gate inputs. Given such a graph, we can answer questions such as ``How long does it take for the signals to propagate from the inputs to the outputs?'' or ``Which gates are on the critical path?''

Finally, Figure gif (d) illustrates that a graph can be used to represent a finite state machine. The points of the graph represent the states and labeled arrows indicate the allowable state transitions. Given such a graph, we can answer questions such as ``Are all the states reachable?'' or ``Can the finite state machine deadlock?''

This chapter is a brief introduction to the body of knowledge known as graph theory . It covers the most common data structures for the representation of graphs and introduces some fundamental graph algorithms.

  • Basics
  • Implementing Graphs
  • Graph Traversals
  • Shortest-Path Algorithms
  • Minimum-Cost Spanning Trees
  • Application: Critical Path Analysis
  • Exercises
  • Projects


Basics

  • Directed Graphs
  • Terminology
  • More Terminology
  • Directed Acyclic Graphs
  • Undirected Graphs
  • Terminology
  • Labeled Graphs
  • Representing Graphs


Directed Graphs

We begin with the definition of a directed graph:

Definition (Directed Graph) A directed graph , or digraph , is an ordered pair [image] with the following properties:
  1. The first component, [image], is a finite, non-empty set. The elements of [image] are called the vertices of G.
  2. The second component, [image], is a finite set of ordered pairs of vertices. I.e., [image]. The elements of [image] are called the edges of G.

For example, consider the directed graph [image] comprised of four vertices and six edges:

The graph G can be represented graphically as shown in Figure gif . The vertices are represented by appropriately labeled circles, and the edges are represented by arrows that connect associated vertices.

figure48375
Figure: A Directed Graph

Notice that because the pairs that represent edges are ordered, the two edges (a,c) and (c,a) are distinct. Furthermore, since [image] is a mathematical set, it cannot contain more than one instance of a given edge. And finally, an edge such as (d,d) may connect a node to itself.


Terminology

Consider a directed graph [image] as given by Definition gif .

  • Each element of [image] is called a vertex or a node of G. Hence, [image] is the set of vertices (or nodes) of G.
  • Each element of [image] is called an edge or an arc of G. Hence, [image] is the set of edges (or arcs) of G.
  • An edge [image] can be represented as [image]. An arrow that points from v to w is known as a directed arc . Vertex w is called the head of the arc because it is found at the arrow head. Conversely, v is called the tail of the arc. Finally, vertex w is said to be adjacent to vertex v.
  • An edge e=(v,w) is said to emanate from vertex v. We use notation [image] to denote the set of edges emanating from vertex v. I.e.,
  • The out-degree of a node is the number of edges emanating from that node. Therefore, the out-degree of v is
  • An edge e=(v,w) is said to be incident on vertex w. We use notation [image] to denote the set of edges incident on vertex w. I.e.,
  • The in-degree of a node is the number of edges incident on that node. Therefore, the in-degree of w is

For example, Table gif enumerates the sets of emanating and incident edges and the in- and out-degrees for each of the vertices in graph [image] shown in Figure gif .

Table: Emanating and Incident Edge Sets for Graph [image] in Figure gif
vertex v | out-degree | in-degree
a | 2 | 1
b | 1 | 1
c | 2 | 2
d | 1 | 2

There is still more terminology to be introduced, but in order to do that, we need the following definition:

Definition (Path and Path Length)

A path in a directed graph [image] is a non-empty sequence of vertices

where [image] for [image] such that [image] for [image]. The length of path P is k-1.

For example, consider again the graph [image] shown in Figure gif . Among the paths contained in [image] there is the path of length zero, [image]; the path of length one, [image]; the path of length two, [image]; and so on. In fact, this graph generates an infinite number of paths! (To see how this is possible, consider that [image] is a path in [image]). Notice too the subtle distinction between a path of length zero, say [image], and the path of length one [image].


More Terminology

Consider the path [image] in a directed graph [image].

  • Vertex [image] is the successor of vertex [image] for [image]. Each element [image] of path P (except the last) has a successor.
  • Vertex [image] is the predecessor of vertex [image] for [image]. Each element [image] of path P (except the first) has a predecessor.
  • A path P is called a simple path if and only if [image] for all i and j such that [image]. However, it is permissible for [image] to be the same as [image] in a simple path.
  • A cycle is a path P of non-zero length in which [image]. The length of a cycle is just the length of the path P.
  • A loop is a cycle of length one. I.e., it is a path of the form [image].
  • A simple cycle is a path that is both a cycle and simple.

Referring again to graph [image] in Figure gif we find that the path [image] is a simple path of length three. Conversely, the path [image] also has length three but is not simple because vertex c occurs twice in the sequence (but not at the ends). The graph contains the path [image] which is a cycle of length three, as well as [image], a cycle of length four. The former is a simple cycle but the latter is not.


Directed Acyclic Graphs

For certain applications it is convenient to deal with graphs that contain no cycles. For example, a tree (see Chapter gif ) is a special kind of graph that contains no cycles.

Definition (Directed Acyclic Graph (DAG))

A directed, acyclic graph is a directed graph that contains no cycles.

Obviously, all trees are DAGs. However, not all DAGs are trees. For example consider the two directed, acyclic graphs, [image] and [image], shown in Figure gif . Clearly [image] is a tree but [image] is not.

figure48874
Figure: Two Directed, Acyclic Graphs


Undirected Graphs

An undirected graph is a graph in which the nodes are connected by undirected arcs . An undirected arc is an edge that has no arrow. Both ends of an undirected arc are equivalent--there is no head or tail. Therefore, we represent an edge in an undirected graph as a set rather than an ordered pair:

Definition (Undirected Graph) An undirected graph is an ordered pair [image] with the following properties:
  1. The first component, [image], is a finite, non-empty set. The elements of [image] are called the vertices of G.
  2. The second component, [image], is a finite set of sets. Each element of [image] is a set that is comprised of exactly two (distinct) vertices. The elements of [image] are called the edges of G.

For example, consider the undirected graph [image] comprised of four vertices and four edges:

The graph [image] can be represented graphically as shown in Figure gif . The vertices are represented by appropriately labeled circles, and the edges are represented by lines that connect associated vertices.

figure49029
Figure: An Undirected Graph

Notice that because an edge in an undirected graph is a set, [image], and since [image] is also a set, it cannot contain more than one instance of a given edge. Another consequence of Definition gif is that there cannot be an edge from a node to itself in an undirected graph because an edge is a set of size two and a set cannot contain duplicates.


Example-Geometric Series Summation Yet Again

In this example we consider the problem of computing a geometric series summation for the last time. We have already seen two algorithms to compute this summation in Sections gif and gif (Programs gif and gif ).

An algorithm to compute a geometric series summation using the closed-form expression (Equation gif ) is given in Program gif . This algorithm makes use of Program gif to compute [image].

program1086
Program: Program to compute [image] using the closed-form expression

To determine the average running time of Program gif we will make use of Equation gif , which gives the average running time for the Power function which is called on line 5. In this case, the arguments are x and n+1, so the running time of the call to Power is [image]. Adding to this the additional work done on line 5 gives the average running time for Program gif :

The running times of the three programs which compute the geometric series summation presented in this chapter are tabulated below in Table gif and are plotted for [image] in Figure gif . Figure gif shows that, according to our simplified model of the computer, for n<4, Program gif has the best running time. However as n increases, Program gif is clearly the fastest of the three and Program gif is the slowest for all values of n.

Table: Running Times of Programs gif , gif and gif
program | T(n)
Program gif |
Program gif | 13n+22
Program gif |

figure1123
Figure: Plot of Running Time vs. n for Programs gif , gif and gif


Terminology

Consider an undirected graph [image] as given by Definition gif .

  • An edge [image] emanates from and is incident on both vertices v and w.
  • The set of edges emanating from a vertex v is the set [image] The set of edges incident on a vertex w is [image].


Labeled Graphs

Practical applications of graphs usually require that they be annotated with additional information. Such information may be attached to the edges of the graph and to the nodes of the graph. A graph which has been annotated in some way is called a labeled graph . Figure gif shows two examples of this.

figure49169
Figure: Labeled Graphs

For example, we can use a directed graph with labeled vertices such as [image] in Figure gif to represent a finite state machine. Each vertex corresponds to a state of the machine and each edge corresponds to an allowable state transition. In such a graph we can attach a label to each vertex that records some property of the corresponding state such as the latency time for that state.

We can use an undirected graph with labeled edges such as [image] in Figure gif to represent geographic information. In such a graph, the vertices represent geographic locations and the edges represent possible routes between locations. In such a graph we might use a label on each edge to represent the distance between the end points.


Representing Graphs

Consider a directed graph [image]. Since [image], graph G contains at most [image] edges. There are [image] possible sets of edges for a given set of vertices [image]. Therefore, the main concern when designing a graph representation scheme is to find a suitable way to represent the set of edges.

  • Adjacency Matrices
  • Sparse vs. Dense Graphs
  • Adjacency Lists


Adjacency Matrices

Consider a directed graph [image] with n vertices, [image]. The simplest graph representation scheme uses an [image] matrix A of zeroes and ones given by

I.e., the [image] element of the matrix, is a one only if [image] is an edge in G. The matrix A is called an adjacency matrix .

For example, the adjacency matrix for graph [image] in Figure gif is

Clearly, the number of ones in the adjacency matrix is equal to the number of edges in the graph.

One advantage of using an adjacency matrix is that it is easy to determine the sets of edges emanating from and incident on a given vertex. E.g., consider vertex [image]. Each one in the [image] row corresponds to an edge that emanates from vertex [image]. Conversely, each one in the [image] column corresponds to an edge incident on vertex [image].

We can also use adjacency matrices to represent undirected graphs. I.e., we represent an undirected graph [image] with n vertices, using an [image] matrix A of zeroes and ones given by

Since the two sets [image] and [image] are equivalent, matrix A is symmetric about the diagonal. I.e., [image]. Furthermore, all of the entries on the diagonal are zero. I.e., [image] for [image].

For example, the adjacency matrix for graph [image] in Figure gif is

In this case, there are twice as many ones in the adjacency matrix as there are edges in the undirected graph.

A simple variation allows us to use an adjacency matrix to represent an edge-labeled graph. For example, given numeric edge labels, we can represent a graph (directed or undirected) using an [image] matrix A in which the [image] is the numeric label associated with edge [image] in the case of a directed graph, and edge [image], in an undirected graph.

For example, the adjacency matrix for the graph [image] in Figure gif is

In this case, the array entries corresponding to non-existent edges have all been set to [image]. Here [image] serves as a kind of sentinel . The value to use for the sentinel depends on the application. For example, if the edges represent routes between geographic locations, then a route of length [image] is much like one that does not exist.

Since the adjacency matrix has [image] entries, the amount of spaced needed to represent the edges of a graph is [image], regardless of the actual number of edges in the graph. If the graph contains relatively few edges, e.g., if [image], then most of the elements of the adjacency matrix will be zero (or [image]). A matrix in which most of the elements are zero (or [image]) is a sparse matrix .


Sparse vs. Dense Graphs

Informally, a graph with relatively few edges is sparse, and a graph with many edges is dense. The following definition defines precisely what we mean when we say that a graph ``has relatively few edges'':

Definition (Sparse Graph) A sparse graph is a graph [image] in which [image].

For example, consider a graph [image] with n nodes. Suppose that the out-degree of each vertex in G is some fixed constant k. Graph G is a sparse graph because [image].

A graph that is not sparse is said to be dense:

Definition (Dense Graph) A dense graph is a graph [image] in which [image].

For example, consider a graph [image] with n nodes. Suppose that the out-degree of each vertex in G is some fraction f of n, [image]. E.g., if n=16 and f=0.25, the out-degree of each node is 4. Graph G is a dense graph because [image].


Adjacency Lists

One technique that is often used for a sparse graph, say [image], uses [image] linked lists--one for each vertex. The linked list for vertex [image] contains the elements of [image], the set of nodes adjacent to [image]. As a result, the lists are called adjacency lists .

Figure gif shows the adjacency lists for the directed graph [image] of Figure gif and the directed graph [image] of Figure gif . Notice that the total number of list elements used to represent a directed graph is [image] but the number of lists elements used to represent an undirected graph is [image]. Therefore, the space required for the adjacency lists is [image].

figure49405
Figure: Adjacency Lists

By definition, a sparse graph has [image]. Hence the space required to represent a sparse graph using adjacency lists is [image]. Clearly this is asymptotically better than using adjacency matrices which require [image] space.


Implementing Graphs

In keeping with the design framework used throughout this text, we view graphs as specialized containers. Formally, the graph [image] is an ordered pair comprised of two sets--a set of vertices and a set of edges. Informally, we can view a graph as a container with two compartments, one which holds vertices and one which holds edges. Therefore, there are three kinds of objects--vertices, edges, and graphs. Accordingly, we need three object classes: Vertex, Edge, and Graph. (See Figure gif ).

figure49670
Figure: Object Class Hierarchy

  • Implementing Vertices
  • Implementing Edges
  • Abstract Graphs and Digraphs
  • Implementing Undirected Graphs
  • Edge-Weighted and Vertex-Weighted Graphs
  • Comparison of Graph Representations


Implementing Vertices

What exactly is a vertex? The answer to this question depends on the application. At the very minimum, every vertex in a graph must be distinguishable from every other vertex in that graph. We can do this by numbering consecutively the vertices of a graph.

Program gif defines the concrete class Vertex. Since we intend to insert instances of this class into containers, the Vertex class is derived from the Object base class. The Vertex class declares a single member the single member variable number. Each vertex inserted into a graph is required to have a different number.

program49684
Program: Vertex Class Definition

A single Vertex constructor is declared. This constructor takes as its lone argument the number that is to be assigned to the vertex. In addition, the cast-to-Number member function (operator Number) is also declared. This allows the programmer to use a vertex wherever an integer is expected. The cast returns the value of the number member variable.


Implementing Edges

An edge in a directed graph is an ordered pair of vertices; an edge in an undirected graph is a set of two vertices. Because of the similarity of these concepts, we use the same class for both--the context in which an edge is used determines whether it is directed or undirected.

Program gif declares the concrete class Edge. Since we intend to insert instances of this class into containers, the Edge class is derived from the Object base class.

program49704
Program: Edge Class Definition

The Edge class contains two member variables--v0 and v1. Each is a reference to a vertex. When an instance of this class is used to represent a directed edge, then the edge represented shall be [image]. I.e., v1 is the head and v0 is the tail. Alternatively, when an Edge instance represents an undirected edge, that edge is simply [image].

Program gif declares a constructor which takes two arguments, both of them references to Vertex instances. The effect of this constructor is to initialize the two corresponding member variables accordingly.

In addition to the constructor, there are three public member functions: V0, V1 and Mate. All three of them are accessors. The first are used to access the member variables v0 and v1, respectively.

For every instance e of the Edge class, the Mate member function satisfies the following identities:

Therefore, if we know that a vertex v is one of the vertices of e, then we can find the other vertex by calling e.Mate(v).


Abstract Graphs and Digraphs

Directed graphs and undirected graphs have many common characteristics. Therefore, it seems reasonable to make use of inheritance in the implementation of graph classes. However, there are several possible ways in which to organize the class hierarchy:

Option A
We could use a common abstract base class Graph to encapsulate the common attributes of all graphs and then derive two different classes UndirectedGraph and DirectedGraph from the common base class.
Option B
Use DirectedGraph as the base class and derive the class UndirectedGraph from it. In this scenario we view an undirected graph as a directed graph that has its arrows removed.
Option C
Use UndirectedGraph as the base class and derive the class DirectedGraph from it. In this scenario we view a directed graph as an undirected graph with arrows added.

As shown in Figure gif , we have chosen to use option C. We have chosen this approach because many algorithms for undirected graphs can also be used with directed graphs. On the other hand, it is often the case that algorithms for directed graphs cannot be used with undirected graphs.

Program gif declares two abstract classes, Graph and Digraph. The Graph class is the base class from which concrete classes that represent undirected graphs are derived. The Digraph abstract class extends the Graph class. The Digraph class is the base class from which concrete classes that represent directed graphs are derived.

program49769
Program: Graph and Digraph Class Definitions

The Graph class is an abstract base class since it contains pure virtual member functions. The graph class adds two member variables to those inherited from the base classes--numberOfEdges and numberOfVertices. A default constructor for the Graph class is also declared. The effect of this constructor is to initialize both member variables with the value zero. Therefore, a graph is initially empty--it contains neither vertices nor edges.

A number of Graph class member functions are declared in Program gif . There are essentially three groups of member functions: accessors and mutators, iterators, and traversals. The operations performed by the member functions are explained in the following sections.

  • Accessors and Mutators
  • Iterators
  • Graph Traversals


Exercises

  1. Determine the running times predicted by the detailed model of the computer given in Section gif for each of the following program fragments:
    1. for (unsigned int i = 0; i < n; ++i)
    ++k;
    2. for (unsigned int i = 1U; i < n; i *= 2U)
    ++k;
    3. for (unsigned int i = n - 1U; i != 0; i /= 2U)
    ++k;
    4. for (unsigned int i = 0; i < n; ++i)
    if (i % 2 == 0)
        ++k;
    5. for (unsigned int i = 0; i < n; ++i)
    for (unsigned int j = 0; j < n; ++j)
        ++k;
    6. for (unsigned int i = 0; i < n; ++i)
    for (unsigned int j = i; j < n; ++j)
        ++k;
    7. for (unsigned int i = 0; i < n; ++i)
    for (unsigned int j = 0; j < i * i; ++j)
        ++k;
  2. Repeat Exercise gif , this time using the simplified model of the computer given in Section gif .
  3. Prove by induction the following summation formulas:
  4. Evaluate each of the following series summations:
  5. Show that [image], for [image]. Hint: Let [image] and show that [image].
  6. Show that [image]. Hint: Let [image] and show that the difference [image] is (approximately) a geometric series summation.
  7. Solve each of the following recurrences by repeated substitution:


Accessors and Mutators

The Graph class declares the following accessor and mutator member functions:

NumberOfEdges
This accessor returns the number of edges contained by the graph.
NumberOfVertices
This accessor returns the number of vertices contained by the graph.
AddVertex
This mutator inserts a given vertex into a graph. For simplicity, we shall assume that a given vertex is inserted into exactly one graph. All the vertices contained in a given graph must have a unique vertex number. Furthermore, if a graph contains n vertices, those vertices shall be numbered 0, 1, ..., n-1. Therefore, the next vertex inserted into the graph shall have the number n.
SelectVertex
This accessor takes an integer, say i where [image], and returns reference to the [image] vertex contained in the graph.
operator[]
This subscript operator takes an integer-valued subscript expression. We shall assume that the default behavior of this operator is to call SelectVertex.
AddEdge
This mutator inserts a given edge into a graph. For simplicity, we shall assume that a given edge is inserted into exactly one graph. Both vertices referenced by the edge must be vertices in the given graph.
IsEdge
This Boolean-valued accessor takes two Vertex::Number arguments. It returns true if the graph contains an edge that connects the corresponding vertices.
SelectEdge
This accessor takes two Vertex::Number arguments. It returns a reference to the edge instance (if it exists) that connects the corresponding vertices. The behavior of this routine is undefined when the edge does not exist. (An implementation will typically throw an exception).
IsCyclic
This Boolean-valued accessor returns true if the graph is cyclic.
IsConnected
This Boolean-valued accessor returns true if the graph is connected. Connectedness of graphs is discussed in Section gif .


Iterators

All the other container classes considered in this text have only one associated iterator. When dealing with graphs, it is convenient to have at least four iterators: The first two are used to enumerate the sets [image] and [image], and the last two are used to enumerate the sets [image] and [image] associated with a given vertex v.

Each of the following member functions allocates dynamically a new iterator instance as follows:

Vertices
This member function of the Graph class returns an iterator that can be used to traverse the elements of [image].
Edges
This member function of the Graph class returns an iterator that can be used to traverse the elements of [image].
IncidentEdges
This member function takes a reference to a vertex, say v, and returns an iterator that can be used to traverse the elements of [image]. The vertex v must be contained by the graph.
EmanatingEdges
This member function takes a reference to a vertex, say v, and returns an iterator that can be used to traverse the elements of [image]. The vertex v must be contained by the graph.


Graph Traversals

These graph traversal functions are analogous to the Accept member function of the container class (see Section gif ). Each of these functions takes a reference to a Visitor and performs a traversal. I.e., all the vertices of the graph are visited systematically. When a vertex is visited, the Visit function of the visitor is applied to that vertex.

DepthFirstTraversal and BreadthFirstTraversal
Both of these functions accept two arguments--a reference to a Visitor and a reference to a vertex. The vertex specifies the starting node for either a depth-first traversal or a breadth-first traversal of the graph.
TopologicalOrderTraversal
A topological sort is an ordering of the nodes of a directed graph. This traversal visits the nodes of a directed graph in the order specified by a topological sort. Therefore, it is a member function of the Digraph class.

The graph traversal algorithms are discussed in Section gif .


Implementing Undirected Graphs

This section describes two concrete classes--GraphAsMatrix and GraphAsLists. These classes both represent undirected graphs. The GraphAsMatrix class represents the edges of a graph using an adjacency matrix. The GraphAsLists class represents the edges of a graph using adjacency lists.

  • Using Adjacency Matrices
  • Using Adjacency Lists


Using Adjacency Matrices

The GraphAsMatrix class is declared in Program gif . The GraphAsMatrix class is a concrete class derived from the base class Graph which is shown in Program gif . Since GraphAsMatrix is a concrete class, it must provide implementations for all the member functions declared as pure virtual functions in the base classes--the function prototypes have been elided for the sake of brevity.

program49884
Program: GraphAsMatrix Class Definition

Each instance of the GraphAsMatrix class represents an undirected graph, say [image]. The two member variables, vertices and adjacencyMatrix, are used to represent the sets [image] and [image], respectively.

The set of vertices, [image], is represented using a one-dimensional array of pointers to Vertex instances. The implementation uses the Array<T> class given in Section gif . The set of edges, [image], is represented using a two-dimensional matrix of pointers to Edge instances. The implementation uses the Array2D<T> class given in Section gif .

The GraphAsMatrix constructor takes a single argument of type unsigned int that specifies the maximum number of vertices that the graph may contain. This quantity specifies the length of the array of vertices and the dimensions of the adjacency matrix. The implementation of the GraphAsMatrix class is left as programming project for the reader (Project gif ).


Using Adjacency Lists

Program gif declares the GraphAsLists concrete class. The GraphAsLists class is derived from the abstract base class Graph. The GraphAsLists class represents the edges of a graph using adjacency lists.

program49914
Program: GraphAsLists Class Definition

Two member variables are declared--vertices and adjacencyLists. The former is an array of pointers to Vertex instances. This array is used to represent the elements of the vertex set [image].

The second member variable is an array of linked lists of pointers to edges. The [image] linked list, adjacencyLists[i], represents the set [image] which is the set of edges emanating from vertex [image]. The implementation uses the LinkedList<T> class given in Section gif .

The GraphAsLists constructor takes a single argument of type unsigned int that specifies the maximum number of vertices that the graph may contain. This quantity specifies the lengths of the array of vertices and the array of adjacency lists. The implementation of the GraphAsLists class is left as programming project for the reader (Project gif ).


Edge-Weighted and Vertex-Weighted Graphs

An edge-weighted graph is a graph in which each edge has been assigned a weight. Similarly, a vertex-weighted graph is a graph in which each vertex has been assigned a weight. In such graphs, the quantity represented by a weight depends on the application.

Since a graph is a container that contains specifically vertices and edges, it is not necessary to introduce new graph classes to represent edge-weighted and vertex-weighted graphs. Instead, we introduce two classes, WeightedVertex and WeightedEdge. By using instances of these new classes together with the existing graph classes, we can create arbitrary edge-weighted and vertex-weighted graphs. Program gif contains the declarations of the WeightedVertex and WeightedEdge classes.

program49950
Program: WeightedVertex and WeightedEdge Class Definitions

Since the WeightedVertex class is derived from the Vertex class, instances of the former can be used wherever the latter is expected. Therefore, we can create a vertex-weighted graph simply by inserting weighted vertices into an instance of one of the graph implementations discussed in the preceding section.

The WeightedVertex<T> class contains a single member variable called weight which is a pointer to an Object instance. Therefore, any object type can be used as the weight. The constructor sets the weight to a given value, and the member function Weight is used to access the weight.

The WeightedEdge<T> class is implemented similarly. I.e., weighted edge instances can be used wherever an edge is expected. By inserting weighted edges into a graph, we get an edge-weighted graph. Clearly, if the application demands it, it is possible to use both weighted vertices and weighted edges in the same graph.


Comparison of Graph Representations

In order to make the appropriate choice when selecting a graph representation scheme, it is necessary to understand the time/space trade-offs. Although the details of the implementations have been omitted, we can still make meaningful conclusions about the performance that we can expect from those implementations. In this section we consider the space required as well as the running times for basic graph operations.

  • Space Comparison
  • Time Comparison


Space Comparison

Consider the representation of a directed graph [image]. In addition to the [image] Vertex class instances and the [image] Edge class instances contained by the graph, there is the storage required by the adjacency matrix. In this case, the matrix is a [image] matrix of pointers to Edge instances. Therefore, the amount of storage required by an adjacency matrix implementation is

On the other hand, consider the amount of storage required when we represent the same graph using adjacency lists. In addition to the vertices and the edges themselves, there are [image] linked lists. If we use the LinkedList class defined in Section gif , each such list has a head and tail pointer. Altogether there are [image] linked lists elements, each of which consists of a pointer to the next element of the lists and a pointer to an edge. Therefore, the total space required is

Notice that the space for the vertices and edges themselves cancels out when we compare Equation gif with Equation gif . If we assume that all pointers require the same amount of space, we can conclude that adjacency lists use less space than adjacency matrices when

For example, given a 10 node graph, the adjacency lists version uses less space when there are fewer than 45 edges. As a rough rule of thumb, we can say that adjacency lists use less space when the average degree of a node, [image], satisfies [image].


Time Comparison

The following four operations are used extensively in the implementations of many different graph algorithms:

find edge (v,w)
Given vertices v and w, this operation locates the corresponding Edge instance. When using an adjacency matrix, we can find an edge in constant time.

When adjacency lists are used, the worst-case running time is [image], since [image] is the length of the adjacency list associated with vertex v.

This is the operation performed by the SelectEdge member function of the Graph class.

enumerate all edges
In order to locate all the edges in when using adjacency matrices, it is necessary to examine all [image] matrix entries. Therefore, the worst-case running time needed to enumerate all the edges is [image].

On the other hand, to enumerate all the edges when using adjacency lists requires the traversal of [image] lists. In all there are [image] edges. Therefore the worst case running time is [image].

This operation is performed using the iterator returned by the Edges member function of the Graph class.

enumerate edges emanating from v
To enumerate all the edges that emanate from vertex v requires a complete scan of the [image] row of an adjacency matrix. Therefore, the worst-case running time when using adjacency matrices is [image].

Enumerating the edges emanating from vertex v is a trivial operation when using adjacency lists. All we need do is traverse the [image] list. This takes [image] time in the worst case.

This operation is performed using the iterator returned by the EmanatingEdges member function of the Graph class.

enumerate edges incident on w
To enumerate all the edges are incident on vertex w requires a complete scan of the [image] column of an adjacency matrix. Therefore, the worst-case running time when using adjacency matrices is [image].

Enumerating the edges incident on vertex w is a non-trivial operation when using adjacency lists. It is necessary to search every adjacency list in order to find all the edges incident on a given vertex. Therefore, the worst-case running time is [image].

This operation is performed using the iterator returned by the IncidentEdges member function of the Graph class.

Table gif summarizes these running times.

Table: Comparison of Graph Representations
 | representation scheme
operation | adjacency matrix | adjacency list
find edge (v,w) | O(1) |
enumerate all edges |  |
enumerate edges emanating from v |  |
enumerate edges incident on w |  |


Projects

  1. Write a non-recursive routine to compute the factorial of n according to Equation gif . Calculate the running time predicted by the detailed model given in Section gif and the simplified model given in Section gif .
  2. Write a non-recursive routine to compute [image] according to Equation gif . Calculate the running time predicted by the detailed model given in Section gif and the simplified model given in Section gif .
  3. Write a program that determines the values of the timing parameters of the detailed model ( [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], and [image]) for the machine on which it is run.
  4. Using the program written for Project gif , determine the timing parameters of the detailed model for your computer. Then, measure the actual running times of Programs gif , gif and gif and compare the measured results with those predicted by Equations gif , gif and gif (respectively).
  5. Given a sequence of n integers, [image], and a small positive integer k, write an algorithm to compute

    without multiplication. Hint: Use Horner's rule and bitwise shifts.

  6. Verify Equation gif experimentally as follows: Generate a large number of random sequences of length n, [image]. For each sequence, test the hypothesis that the probability that [image] is larger than all its predecessors in the sequence is [image]. (For a good source of random numbers, see Section gif ).


Graph Traversals

There are many different applications of graphs. As a result, there are many different algorithms for manipulating them. However, many of the different graph algorithms have in common the characteristic that they systematically visit all the vertices in the graph. I.e., the algorithm walks through the graph data structure and performs some computation at each vertex in the graph. This process of walking through the graph is called a graph traversal .

While there are many different possible ways in which to systematically visit all the vertices of a graph, certain traversal methods occur frequently enough that they are given names of their own. This section presents three of them--depth-first traversal, breadth-first traversal and topological sort.

  • Depth-First Traversal
  • Breadth-First Traversal
  • Topological Sort
  • Graph Traversal Applications:
    Testing for Cycles and Connectedness


Depth-First Traversal

The depth-first traversal of a graph is like the depth-first traversal of a tree discussed in Section gif . A depth-first traversal of a tree always starts at the root of the tree. Since a graph has no root, when we do a depth-first traversal, we must specify the vertex at which to begin.

A depth-first traversal of a tree visits a node and then recursively visits the subtrees of that node. Similarly, depth-first traversal of a graph visits a vertex and then recursively visits all the vertices adjacent to that node. The catch is that the graph may contain cycles, but the traversal must visit every vertex at most once. The solution to the problem is to keep track of the nodes that have been visited, so that the traversal does not suffer the fate of infinite recursion.

figure50046
Figure: Depth-First Traversal

For example, Figure gif illustrates the depth-first traversal of the directed graph [image] starting from vertex c. The depth-first traversal visits the nodes in the order

A depth-first traversal only follows edges that lead to unvisited vertices. As shown in Figure gif , if we omit the edges that are not followed, the remaining edges form a tree. Clearly, the depth-first traversal of this tree is equivalent to the depth-first traversal of the graph

  • Implementation
  • Running Time Analysis


Implementation

Program gif gives the code for the two DepthFirstTraversal routines of the Graph class. One of them accepts two arguments, the other, three. As indicated in Program gif , the two-argument routine is declared public whereas the three-argument one is protected.

The user of the Graph class only sees the two-argument DepthFirstTraversal routine. This routine takes a reference to a Visitor instance and a reference to a Vertex instance. The idea is that the Visit function of the visitor is called once for each vertex in the graph and the vertices are visited in depth-first traversal order starting from the specified vertex.

program50318
Program: Graph Class DepthFirstTraversal Member Function Definition

In order to ensure that each vertex is visited at most once, an array of length [image] of Boolean values called visited is used (line 4). I.e., [image] only if vertex i has been visited. All the array elements are initially false (lines 5-6). After initializing the array, the two-argument routine calls the three-argument one, passing it a reference to the array as the third argument.

The three-argument routine returns immediately if the visitor is done. Otherwise, it visits the specified node, and then it follows all the edges emanating from that node and recursively visits the adjacent vertices if those vertices have not already been visited.


Running Time Analysis

The running time of the depth-first traversal routine depends on the graph representation scheme used. The traversal visits each node in the graph at most once. When a node is visited, all the edges emanating from that node are considered. During a complete traversal enumerates every edge in the graph.

Therefore, the worst-case running time for the depth-first traversal of a graph is represented using an adjacency matrix is

When adjacency lists are used, the worst case running time for the depth-first traversal routine is

Recall that for a sparse graph graph [image]. If the sparse graph is represented using adjacency lists and if [image] and [image] the worst-case running time of the depth-first traversal is simply [image].


Breadth-First Traversal

The breadth-first traversal of a graph is like the breadth-first traversal of a tree discussed in Section gif . The breadth-first traversal of a tree visits the nodes in the order of their depth in the tree. Breadth-first tree traversal first visits all the nodes at depth zero (i.e., the root), then all the nodes at depth one, and so on.

Since a graph has no root, when we do a breadth-first traversal, we must specify the vertex at which to start the traversal. Furthermore, we can define the depth of a given vertex to be the length of the shortest path from the starting vertex to the given vertex. Thus, breadth-first traversal first visits the starting vertex, then all the vertices adjacent to the starting vertex, and the all the vertices adjacent to those, and so on.

Section gif presents a non-recursive breadth-first traversal algorithm for N-ary trees that uses a queue to keep track vertices that need to be visited. The breadth-first graph traversal algorithm is very similar.

First, the starting vertex is enqueued. Then, the following steps are repeated until the queue is empty:

  1. Remove the vertex at the head of the queue and call it vertex.
  2. Visit vertex.
  3. Follow each edge emanating from vertex to find the adjacent vertex and call it to. If to has not already been put into the queue, enqueue it.
Notice that a vertex can be put into the queue at most once. Therefore, the algorithm must somehow keep track of the vertices that have been enqueued.

figure50346
Figure: Breadth-First Traversal

Figure gif illustrates the breadth-first traversal of the directed graph [image] starting from vertex a. The algorithm begins by inserting the starting vertex, a, into the empty queue. Next, the head of the queue (vertex a) is dequeued and visited, and the vertices adjacent to it (vertices b and c) are enqueued. When, b is dequeued and visited we find that there is only adjacent vertex, c, and that vertex is already in the queue. Next vertex c is dequeued and visited. Vertex c is adjacent to a and d. Since a has already been enqueued (and subsequently dequeued) only vertex d is put into the queue. Finally, vertex d is dequeued an visited. Therefore, the breadth-first traversal of [image] starting from a visits the vertices in the sequence

  • Implementation
  • Running Time Analysis


Implementation

Program gif gives the code for the BreadthFirstTraversal routine of the Graph class. This routine takes a reference to a Visitor instance and a reference to a Vertex instance. The Visit function of the visitor is called once for each vertex in the graph and the vertices are visited in breadth-first traversal order starting from the specified vertex.

program50531
Program: Graph Class BreadthFirstTraversal Member Function Definition

An Boolean-valued array, enqueued, is used to keep track of the vertices that have been put into the queue. The elements of the array are all initialized to false (lines 4-6).

Next, a new queue instance is allocated dynamically (line 8). The queue is used to contain only vertices. Since the vertices are owned by the graph, they cannot also be owned by the queue. Therefore, the RescindOwnership function of the queue is called (line 9). This way, when the queue destructor is called, it will not attempt to delete any of the its contained objects.

The second argument of the BreadthOrderTraversal function is a reference to a const vertex. Therefore, the routine must not modify the start vertex. However, we need to push the vertex onto the queue and since the Enqueue function takes a non-const Object reference, it is necessary to cast away the constness using a constcast (line 12). This is not an unsafe cast in this context, because the queue is a local variable of the BreadthOrderTraversal routine and because the routine does not modify anything which it later dequeues from the queue.

The main loop of the BreadthFirstTraversal routine comprises lines 13-30. This loop continues as long as there is a vertex in the queue and the visitor is willing to do more work (line 13). In each iteration exactly one vertex is dequeued and visited (lines 15-17). After a vertex is visited, all the successors of that node are examined (lines 18-21). Every successor of the node that has not yet been enqueued is put into the queue and the fact that it has been enqueued is recored in the array enqueued (lines 22-26).


Running Time Analysis

The breadth-first traversal enqueues each node in the graph at most once. When a node is dequeued, all the edges emanating from that node are considered. Therefore, a complete traversal enumerates every edge in the graph.

The actual running time of the breadth-first traversal routine depends on the graph representation scheme used. The worst-case running time for the traversal of a graph represented using an adjacency matrix is

When adjacency lists are used, the worst case running time for the breadth-first traversal routine is

If the graph is sparse, then [image]. Therefore, if a sparse graph is represented using adjacency lists and if [image], the worst-case running time of the breadth-first traversal is just [image].


Topological Sort

A topological sort is an ordering of the vertices of a directed acyclic graph given by the following definition:

Definition (Topological Sort) Consider a directed acyclic graph [image]. A topological sort of the vertices of G is a sequence [image] in which each element of [image] appears exactly once. For every pair of distinct vertices [image] and [image] in the sequence S, if [image] is an edge in G, i.e., [image], then i<j.

Informally, a topological sort is a list of the vertices of a DAG in which all the successors of any given vertex appear in the sequence after that vertex. Consider the directed acyclic graph [image] shown in Figure gif . The sequence [image] is a topological sort of the vertices of [image]. To see that this is so, consider the set of vertices:

The vertices in each edge are in alphabetical order, and so is the sequence S.

figure50567
Figure: A Directed Acyclic Graph

It should also be evident from Figure gif that a topological sort is not unique. For example, the following are also valid topological sorts of the graph [image]:

One way to find a topological sort is to consider the in-degrees of the vertices. (The number above a vertex in Figure gif is the in-degree of that vertex). Clearly the first vertex in a topological sort must have in-degree zero and every DAG must contain at least one vertex with in-degree zero. A simple algorithm to create the sort goes like this:

Repeat the following steps until the graph is empty:

  1. Select a vertex that has in-degree zero.
  2. Add the vertex to the sort.
  3. Delete the vertex and all the edges emanating from it from the graph.

  • Implementation
  • Running Time Analysis


Implementation

Instead of implementing an algorithm that computes a topological sort, we have chosen to implement a traversal that visits the vertices of a DAG in the order given by the topological sort. The topological order traversal can be used to implement many other graph algorithms. Furthermore, given such a traversal, it is easy to define a visitor that computes a topological sort.

In order to implement the algorithm described in the preceding section, an array of integers of length [image] is used to record the in-degrees of the vertices. As a result, it is not really necessary to remove vertices or edges from the graph during the traversal. Instead, the effect of removing a vertex and all the edges emanating from that vertex is simulated by decreasing the apparent in-degrees of all the successors of the removed vertex.

In addition, we use a queue to keep track of the vertices that have not yet been visited, but whose in-degree is zero. Doing so eliminates the need to search the array for zero entries.

Program gif defines the TopologicalOrderTraversal routine of the Digraph class. This routine takes as its lone argument a reference to a visitor instance. The Visit function of the visitor is called once for each vertex in the graph. The order in which the vertices are visited is given by a topological sort of those vertices.

program50808
Program: Digraph Class TopologicalOrderTraversal Member Function Definition

The algorithm begins by computing the in-degrees of all the vertices. An array of unsigned integers of length [image] called inDegree is used for this purpose. First, all the array elements are set to zero. Then, for each edge [image], array element [image] is increased by one (lines 3-12).

Next, a queue to hold vertices is created. Since the vertices are owned by the graph, the RescindOwnership member function of the queue is called. That way, the queue will not attempt to delete any contained vertices when its destructor is called. After the queue has been initialized, all vertices with in-degree zero are enqueued (lines 14-18).

The main loop of the TopologicalOrderTraversal routine comprises lines 19-33. This loop continues as long as the queue is not empty and the visitor is not finished. In each iteration of the main loop exactly one vertex is dequeued and visited (lines 21-23).

Once a vertex has been visited, the effect of removing that vertex from the graph is simulated by decreasing by one the in-degrees of all the successors of that vertex. When the in-degree of a vertex becomes zero, that vertex is enqueued (lines 24-30).


Running Time Analysis

The topological-order traversal enqueues each node in the graph at most once. When a node is dequeued, all the edges emanating from that node are considered. Therefore, a complete traversal enumerates every edge in the graph.

The worst-case running time for the traversal of a graph represented using an adjacency matrix is

When adjacency lists are used, the worst case running time for the topological-order traversal routine is


Asymptotic Notation

Suppose we are considering two algorithms, A and B, for solving a given problem. Furthermore, let us say that we have done a careful analysis of the running times of each of the algorithms and determined them to be [image] and [image], respectively, where n is a measure of the problem size. Then it should be a fairly simple matter to compare the two functions [image] and [image] to determine which algorithm is the best!

But is it really that simple? What exactly does it mean for one function, say [image], to be better than another function, [image]? One possibility arises if we know the problem size a priori. E.g., suppose the problem size is [image] and [image]. Then clearly algorithm A is better than algorithm B for problem size [image].

In the general case, we have no a priori knowledge of the problem size. However, if it can be shown, say, that [image], then algorithm A is better than algorithm B regardless of the problem size.

Unfortunately, we usually don't know the problem size beforehand, nor is it true that one of the functions is less than or equal the other over the entire range of problem sizes. In this case, we consider the asymptotic behavior of the two functions for very large problem sizes.

  • An Asymptotic Upper Bound-Big Oh
  • An Asymptotic Lower Bound-Omega
  • More Notation-Theta and Little Oh
  • Asymptotic Analysis of Algorithms
  • Exercises
  • Projects


Graph Traversal Applications: Testing for Cycles and Connectedness

 Data Structures and Algorithms with Object-Oriented Design Patterns in C++
next up previous contents index

Graph Traversal Applications:
Testing for Cycles and Connectedness

This section presents several graph algorithms that are based on graph traversals. The first two algorithms test undirected and directed graphs for connectedness. Both algorithms are implemented using the depth-first traversal. The third algorithm tests a directed graph for cycles. It is implemented using a topological-order traversal.

  • Connectedness of an Undirected Graph
  • Connectedness of a Directed Graph
  • Testing for Cycles in a Directed Graph


Connectedness of an Undirected Graph

Definition (Connectedness of an Undirected Graph) An undirected graph [image] is connected if there is a path in G between every pair of vertices in [image].

Consider the undirected graph shown in Figure gif . It is tempting to interpret this figure as a picture of two graphs. However, the figure actually represents the undirected graph [image], given by

Clearly, the graph [image] is not connected. For example, there is no path between vertices a and d. In fact, the graph [image] consists of two, unconnected parts, each of which is a connected sub-graph. The connected sub-graphs of a graph are called connected components .

figure50838
Figure: An Unconnected, Undirected Graph with Two (Connected) Components

A traversal of an undirected graph (either depth-first or breadth-first) starting from any vertex will only visit all the other vertices of the graph if that graph is connected. Therefore, there is a very simply way to test whether an undirected graph is connected: Count the number of vertices visited during a traversal of the graph. Only if all the vertices are visited is the graph connected.

Program gif shows how this can be implemented. The IsConnected member function of the Graph class is a Boolean-valued accessor that returns true if the graph is connected. The routine is implemented using a CountingVisitor and the DepthFirstTraversal routine.

program51016
Program: Graph Class IsConnected Member Function Definition

A CountingVisitor is a visitor that simply counts the number of vertices it visits. It has a single member variable, count, which is initialized to zero in the constructor. The Visit routine adds one the count each time it is called and the Count accessor returns the value of the count.

The worst-case running time of the IsConnected routine is determined by the time taken by the DepthFirstTraversal. Clearly in this case [image]. Therefore, the running time of IsConnected is [image] when adjacency matrices are used to represent the graph and [image] when adjacency lists are used.


Connectedness of a Directed Graph

When dealing with directed graphs, we define two kinds of connectedness, strong and weak. Strong connectedness of a directed graph is defined as follows:

Definition (Strong Connectedness of a Directed Graph) A directed graph [image] is strongly connected if there is a path in G between every pair of vertices in [image].

For example, Figure gif shows the directed graph [image] given by

Notice that the graph [image] is not connected! E.g., there is no path from any of the vertices in [image] to any of the vertices in [image]. Nevertheless, the graph ``looks'' connected in the sense that it is not made of up of separate parts in the way that the graph [image] in Figure gif is.

This idea of ``looking'' connected is what weak connectedness represents. To define weak connectedness we need to introduce first the notion of the undirected graph that underlies a directed graph: Consider a directed graph [image]. The underlying undirected graph is the graph [image] where [image] represents the set of undirected edges that is obtained by removing the arrowheads from the directed edges in G:

figure51054
Figure: An Weakly Connected Directed Graph and the Underlying Undirected Graph

Weak connectedness of a directed graph is defined with respect to its underlying, undirected graph:

Definition (Weak Connectedness of a Directed Graph) A directed graph [image] is weakly connected if the underlying undirected graph [image] is connected.

For example, since the undirected graph [image] in Figure gif is connected, the directed graph [image] is weakly connected. Consider what happens when we remove the edge (b,e) from the directed graph [image]. The underlying undirected graph that we get is [image] in Figure gif . Therefore, when we remove edge (b,e) from [image], the graph that remains is neither strongly connected nor weakly connected.

A traversal of a directed graph (either depth-first or breadth-first) starting from a given vertex will only visit all the vertices of an undirected graph if there is a path from the start vertex to every other vertex. Therefore, a simple way to test whether a directed graph is strongly connected uses [image] traversals--one starting from each vertex in [image]. Each time the number of vertices visited is counted. The graph is strongly connected if all the vertices are visited in each traversal.

Program gif shows how this can be implemented. It shows the IsConnected member function of the Digraph class which returns the Boolean value true if the graph is strongly connected.

program51227
Program: Digraph Class IsConnected Member Function Definition

The routine consists of a loop over all the vertices of the graph. Each iteration does a DepthFirstTraversal using the CountingVisitor given in Program gif . The running time for one iteration is essentially that of the DepthOrderTraversal since [image] for the counting visitor. Therefore, the worst-case running time for the IsConnected routine is [image] when adjacency matrices are used and [image] when adjacency lists are used to represent the graph.


Testing for Cycles in a Directed Graph

The final application of graph traversal that we consider in this section is to test a directed graph for cycles. An easy way to do this is to attempt a topological-order traversal using the algorithm given in Section gif . This algorithm only visits all the vertices of a directed graph if that graph contains no cycles.

To see why this is so, consider the directed cyclic graph [image] shown in Figure gif . The topological traversal algorithm begins by computing the in-degrees of the vertices. (The number shown below each vertex in Figure gif is the in-degree of that vertex).

figure51247
Figure: A Directed Cyclic Graph

At each step of the traversal, a vertex with in-degree of zero is visited. After a vertex is visited, the vertex and all the edges emanating from that vertex are removed from the graph. Notice that if we remove vertex a and edge (a,b) from [image], all the remaining vertices have in-degrees of one. The presence of the cycle prevents the topological-order traversal from completing.

Therefore, the a simple way to test whether a directed graph is cyclic is to attempt a topological traversal of its vertices. If all the vertices are not visited, the graph must be cyclic.

Program gif gives the implementation of the IsCyclic member function of the Digraph class. This Boolean-valued accessor returns true if the graph is cyclic. The implementation simply makes use of a CountingVisitor to count the number of vertices visited during a TopologicalOrderTraversal of the graph.

program51416
Program: Digraph Class IsCyclic Member Function Definition

The worst-case running time of the IsCyclic routine is determined by the time taken by the TopologicalOrderTraversal. Since [image], the running time of IsCyclic is [image] when adjacency matrices are used to represent the graph and [image] when adjacency lists are used.


Shortest-Path Algorithms

In this section we consider edge-weighted graphs, both directed and undirected, in which the weight measures the cost of traversing that edge. The units of cost depend on the application.

For example, we can use a directed graph to represent a network of airports. In such a graph the vertices represent the airports and the edges correspond to the available flights between airports. In this scenario there are several possible cost metrics: If we are interested in computing travel time, then we use an edge-weighted graph in which the weights represent the flying time between airports. If we are concerned with the financial cost of a trip, then the weights on the edges represent the monetary cost of a ticket. Finally, if we are interested the actual distance traveled, then the weights represent the physical distances between airports.

If we are interested in traveling from point A to B, we can use a suitably labeled graph to answer the following questions: What is the fastest way to get from A to B? Which route from A to B has the least expensive airfare? What is the shortest possible distance traveled to get from A to B?

Each of these questions is an instance of the same problem: Given an edge-weighted graph, [image], and two vertices, [image] and [image], find the path that starts at [image] and ends at [image] that has the smallest weighted path length. The weighted length of a path is defined as follows:

Definition (Weighted Path Length) Consider an edge-weighted graph [image]. Let [image] be the weight on the edge connecting [image] to [image]. A path in G is a non-empty sequence of vertices [image]. The weighted path length of path P is given by

The weighted length of a path is the sum of the weights on the edges in that path. Conversely, the unweighted length of a path is simply the number of edges in that path. Therefore, the unweighted length of a path is equivalent to the weighted path length obtained when all edge weights are one.

  • Single-Source Shortest Path
  • All-Pairs Source Shortest Path


Single-Source Shortest Path

In this section we consider the single-source shortest path problem: Given an edge-weighted graph [image] and a vertex [image], find the shortest weighted path from [image] to every other vertex in [image].

Why do we find the shortest path to every other vertex if we are interested only in the shortest path from, say, [image] to [image]? It turns out that in order to find the shortest path from [image] to [image], it is necessary to find the shortest path from [image] to every other vertex in G!

Clearly, when we search for the shortest path, we must consider all the vertices in [image]. If a vertex is ignored, say [image], then we will not consider any of the paths from [image] to [image] that pass through [image]. But if we fail to consider all the paths from [image] to [image], we cannot be assured of finding the shortest one.

Furthermore, suppose the shortest path from [image] to [image] passes through some intermediate node [image]. I.e., the shortest path is of the form [image] It must be the case that the portion of P between [image] to [image] is also the shortest path from [image] to [image]. Suppose it is not. Then there exists another shorter path from [image] to [image]. But then, P would not be the shortest path from [image] to [image], because we could obtain a shorter one by replacing the portion of P between [image] and [image] by the shorter path.

Consider the directed graph [image] shown in Figure gif . The shortest weighted path between vertices b and f is the path [image] which has the weighted path length nine. On the other hand, the shortest unweighted path is from b to f is the path of length three, [image].

figure51449
Figure: Two Edge-Weighted Directed Graphs

As long as all the edge weights are non-negative (as is the case for [image]), the shortest-path problem is well defined. Unfortunately, things get a little tricky in the presence of negative edge weights.

For example, consider the graph [image] shown in Figure gif . Suppose we are looking for the shortest path from d to f. Exactly two edges emanate from vertex d, both with the same edge weight of five. If the graph contained only positive edge weights, there could be no shorter path than the direct path [image].

However, in a graph that contains negative weights, a long path gets ``shorter'' when we add edges with negative weights to it. E.g., the path [image] has a total weighted path length of four, even though the first edge, (d,a), has the weight five.

But negative weights are even more insidious than this: For example, the path [image], which also joins vertex d to f, has a weighted path length of two but the path [image] has length zero. I.e., as the number of edges in the path increases, the weighted path length decreases! The problem in this case is the existence of the cycle [image] the weighted path length of which is less than zero. Such a cycle is sometimes called a negative cost cycle .

Clearly, the shortest-path problem is not defined for graphs that contain negative cost cycles. However, negative edges are not intrinsically bad. Solutions to the problem do exist for graphs that contain both positive and negative edge weights, as long as there are no negative cost cycles. Nevertheless, the problem is greatly simplified when all edges carry non-negative weights.

  • Dijkstra's Algorithm
  • Data Structures for Dijkstra's Algorithm
  • Implementation
  • Running Time Analysis


Dijkstra's Algorithm

Dijkstra's algorithm is a greedy algorithm for solving the single-source, shortest-path problem on an edge-weighted graph in which all the weights are non-negative. It finds the shortest paths from some initial vertex, say [image], to all the other vertices one-by-one. The essential feature of Dijkstra's algorithm is the order in which the paths are determined: The paths are discovered in the order of their weighted lengths, starting with the shortest, proceeding to the longest.

For each vertex v, Dijkstra's algorithm keeps track of three pieces of information, [image], [image], and [image]:

The Boolean-valued flag [image] indicates that the shortest path to vertex v is known. Initially, [image] for all [image].
The quantity [image] is the length of the shortest known path from [image] to v. When the algorithm begins, no shortest paths are known. The distance [image] is a tentative distance. During the course of the algorithm candidate paths are examined and the tentative distances are modified.

Initially, [image] for all [image] such that [image], while [image].

The predecessor of vertex v on the shortest path from [image] to v. I.e., the shortest path from [image] to v has the form [image].

Initially, [image] is unknown for all [image].

Dijkstra's algorithm proceeds in phases. The following steps are performed in each pass:

  1. From the set of vertices for with [image], select the vertex v having the smallest tentative distance [image].
  2. Set [image].
  3. For each vertex w adjacent to v for which [image], test whether the tentative distance [image] is greater than [image]. If it is, set [image] and set [image].
In each pass exactly one vertex has its [image] set to true. The algorithm terminates after [image] passes are completed at which time all the shortest paths are known.

Table gif illustrates the operation of Dijkstra's algorithm as it finds the shortest paths starting from vertex b in graph [image] shown in Figure gif .

Table: Operation of Dijkstra's Algorithm
 | passes
vertex | initially |  |  | 1 |  |  | 2 |  |  | 3 |  |  | 4 |  |  | 5 |  |  | 6
a |  |  |  |  | 3 | b |  | 3 | b |  | 3 | b |  | 3 | b |  | 3 | b |  | 3 | b
b |  | 0 | -- |  | 0 | -- |  | 0 | -- |  | 0 | -- |  | 0 | -- |  | 0 | -- |  | 0 | --
c |  |  |  |  | 5 | b |  | 4 | a |  | 4 | a |  | 4 | a |  | 4 | a |  | 4 | a
d |  |  |  |  |  |  |  |  |  |  | 6 | c |  | 6 | c |  | 6 | c |  | 6 | c
e |  |  |  |  |  |  |  |  |  |  | 8 | c |  | 8 | c |  | 8 | c |  | 8 | c
f |  |  |  |  |  |  |  |  |  |  |  |  |  | 11 | d |  | 9 | e |  | 9 | e

Initially all the tentative distances are [image], except for vertex b which has tentative distance zero. Therefore, vertex b is selected in the first pass. The mark [image] beside an entry in Table gif indicates that the shortest path is known ( [image]).

Next we follow the edges emanating from vertex b, [image] and [image], and update the distances accordingly. The new tentative distances for a becomes 3 and the new tentative distance for c is 5. In both cases, the next-to-last vertex on the shortest path is vertex b.

In the second pass, vertex a is selected and its entry is marked with [image] indicating the shortest path is known. There is one edge emanating from a, [image]. The distance to c via a is 4. Since this is less than the tentative distance to c, vertex c is given the new tentative distance 4 and its predecessor on the shortest-path is set to a. The algorithm continues in this fashion for a total of [image] passes until all the shortest paths have been found.

The shortest-path information contained in the right-most column of Table gif can be represented in the form of a vertex-weighted graph as shown in Figure gif .

figure51779
Figure: The Shortest-Path Graph for

This graph contains the same set of vertices as the problem graph [image]. Each vertex v is labeled with the length [image] of the shortest path from b to v. Each vertex (except b) has a single emanating edge that connects the vertex to the next-to-last vertex on the shortest-path. By following the edges in this graph from any vertex v to vertex b, we can construct the shortest path from b to v in reverse.


Data Structures for Dijkstra's Algorithm

The implementation of Dijkstra's algorithm described below uses the TableEntry structure declared in Program gif . Each TableEntry instance has three fields, known, distance and predecessor, which correspond to the variables [image], [image] and [image], respectively.

program52055
Program: Data Structures for Dijkstra's Algorithm

In each pass of its operation, Dijkstra's algorithm selects from the set of vertices for which the shortest-path is not yet known the one with the smallest tentative distance. Therefore, we use a priority queue to represent this set of vertices.

The priority assigned to a vertex is its tentative distance. The class Assoc defined in Program gif is used to associate of a priority with a given object instance. The Assoc class is derived from the Association class given in Section gif . Notice that the Assoc constructor calls RescindOwnership to prevent the destructor from deleting the associated object instance.


Implementation

A version of Dijkstra's algorithm is shown in Program gif . The DijkstrasAlgorithm function takes two arguments. The first is a const reference to a directed graph instance. It is assumed that the directed graph is an edge-weighted graph in which the weights are instances of the Int class defined in Program gif . The second argument is a const reference to the start node.

program52078
Program: Dijkstra's Algorithm

The DijkstrasAlgorithm routine returns its result in the form of a shortest-path graph. Therefore, the return value is a reference to a Digraph instance. The function allocates the storage, constructs the shortest-path graph and returns a reference to that graph.

The main data structures used are called table and queue (lines 4-5). The former is an array of [image] TableEntry elements. The latter is a reference to a priority queue. In this case, a BinaryHeap of length [image] is used. (See Section gif ).

The algorithm begins by setting the tentative distance for the start vertex to zero and inserting the start vertex into the priority queue with priority zero (lines 7-8).

The main loop of the routine comprises lines 8-35. In each iteration of this loop the vertex with the smallest distance is dequeued (lines 10-12). The vertex is processed only if its table entry indicates that the shortest path is not already known (line 13).

When a vertex v0 is processed, its shortest path is deemed to be known (line 15). Then each vertex v1 adjacent to vertex is considered (lines 16-19). The distance to v1 along the path that passes through v0 is computed (lines 20-23). If this distance is less than the tentative distance associated with v1, entries in the table for v1 are updated, and the v1 is given a new priority and inserted into the priority queue (lines 24-29).

The main loop terminates when all the shortest paths have been found. The shortest-path graph is then constructed using the information in the table (lines 38-45).


Running Time Analysis

The running time of the DijkstrasAlgorithm routine is dominated by the running time of the main loop (lines 8-35). (It is easy to see that lines 3-7 run in constant time, and lines 38-45 run in [image] time).

To determine the running time of the main loop, we proceed as follows: First, we ignore temporarily the time required for the Enqueue and Dequeue operations in the priority queue. Clearly, each vertex in the graph is processed exactly once. When a vertex is processed all the edges emanating from it are considered. Therefore, the time (ignoring the priority queue operations) taken is O(|V|+|E|) when adjacency lists are used and [image] when adjacency matrices are used.

Now, we add back the worst-case time required for the priority queue operations. In the worst case, a vertex is enqueued and subsequently dequeued once for every edge in the graph. Therefore, the length of the priority queue is at most [image]. As a result, the worst-case time for each operation is [image].

Thus, the worst-case running time for Dijkstra's algorithm is

when adjacency lists are used, and

when adjacency matrices are used to represent the input graph.


An Asymptotic Upper Bound-Big Oh

In 1892, P. Bachmann invented a notation for characterizing the asymptotic behavior of functions. His invention has come to be known as big oh notation:

Definition (Big Oh) Consider a function f(n) which is non-negative for all integers [image]. We say that ``f(n) is big oh g(n),'' which we write f(n)=O(g(n)), if there exists an integer [image] and a constant c>0 such that for all integers [image], [image].

  • A Simple Example
  • Big Oh Fallacies and Pitfalls
  • Properties of Big Oh
  • About Polynomials
  • About Logarithms
  • Tight Big Oh Bounds
  • More Big Oh Fallacies and Pitfalls
  • Conventions for Writing Big Oh Expressions


All-Pairs Source Shortest Path

In this section we consider the all-pairs, shortest path problem: Given an edge-weighted graph [image], for each pair of vertices in [image] find the length of the shortest weighted path between the two vertices.

One way to solve this problem is to run Dijkstra's algorithm [image] times in turn using each vertex in [image] as the initial vertex. Therefore, we can solve the all-pairs problem in [image] time when adjacency lists are used, and [image], when adjacency matrices are used. However, for a dense graph ( [image]) the running time of Dijkstra's algorithm is [image], regardless of the representation scheme used.

  • Floyd's Algorithm
  • Implementation
  • Running Time Analysis


Floyd's Algorithm

Floyd's algorithm uses the dynamic programming method to solve the all-pairs shortest-path problem on a dense graph. The method makes efficient use of an adjacency matrix to solve the problem. Consider an edge-weighted graph [image], where C(v,w) represents the weight on edge (v,w). Suppose the vertices are numbered from 1 to [image]. I.e., let [image]. Furthermore, let [image] be the set comprised of the first k vertices in [image]. I.e., [image], for [image].

Let [image] be the shortest path from vertex v to w that passes only through vertices in [image], if such a path exists. I.e., the path [image] has the form

Let [image] be the length of path [image]:

Since [image], the [image] paths are correspond to the edges of G:

Therefore, the [image] path lengths correspond to the weights on the edges of G:

Floyd's algorithm computes the sequence of matrices [image]. The distances in [image] represent paths with intermediate vertices in [image]. Since [image], we can obtain the distances in [image] from those in [image] by considering only the paths that pass through vertex [image]. Figure gif illustrates how this is done.

figure52132
Figure: Calculating [image] in Floyd's Algorithm

For every pair of vertices (v,w), we compare the distance [image], (which represents the shortest path from v to w that does not pass through [image]) with the sum [image] (which represents the shortest path from v to w that does pass through [image]). Thus, [image] is computed as follows:


Implementation

An implementation of Floyd's algorithm is shown in Program gif . The FloydsAlgorithm function takes as its lone argument a const reference to a directed graph instance. The directed graph is assumed to be an edge-weighted graph in which the weights are instances of the Int class defined in Program gif .

program52275
Program: Floyd's Algorithm

The FloydsAlgorithm routine returns its result in the form of an edge-weighted directed graph. Therefore, the return value is a reference to a Digraph instance. The function allocates the storage, constructs the graph and returns a reference to that graph.

The principal data structure use by the algorithm is a [image] matrix of unsigned integers called distance. All the elements of the matrix are initially set to [image] (lines 6-9). Next, an edge iterator is used to visit all the edges in the input graph in order to transfer the weights from the graph to the distance matrix (lines 10-19).

The main work of the algorithm is done in three, nested loops (lines 21-31). The outer loop computes the sequence of distance matrices [image]. The inner two loops consider all possible pairs of vertices. Notice that as [image] is computed, its entries overwrite those of [image].

Finally, the values in the distance matrix are transfered to the result graph. The result graph contains the same set of vertices as the input graph. For each finite entry in the distance matrix, a weighted edge is added to the result graph.


Running Time Analysis

The worst-case running time for Floyd's algorithm is easily determined. Creating and initializing the distance matrix is [image] (lines 6-9). Transferring the weights from the input graph to the distance matrix requires [image] time if adjacency lists are used, and [image] time when an adjacency matrix is used to represent the input graph (lines 10-19).

The running time for the three nested loops is [image] in the worst case. Finally, constructing the result graph and transferring the entries from the distance matrix to the result requires [image] time. As a result, the worst-case running time of Floyd's algorithm is [image].


Minimum-Cost Spanning Trees

In this section we consider undirected graphs and their subgraphs. A subgraph of a graph [image] is any graph [image] such that [image] and [image]. In particular, we consider connected undirected graphs and their minimal subgraphs . The minimal subgraph of a connected graph is called a spanning tree:

Definition (Spanning Tree) Consider a connected, undirected graph [image]. A spanning tree of G is a subgraph of G, say [image], with the following properties:
  1. .
  2. T is connected.
  3. T is acyclic.

Figure gif shows an undirected graph, [image], together with three of its spanning trees. A spanning tree is called a tree because every acyclic undirected graph can be viewed as a general, unordered tree. Because the edges are undirected, any vertex may be chosen to serve as the root of the tree. For example, the spanning tree of [image] given in Figure gif (c) can be viewed as the general, unordered tree

figure52318
Figure: An Undirected Graph and Three Spanning Trees

According to Definition gif , a spanning tree is connected. Therefore, as long as the tree contains more than one vertex, there can be no vertex with degree zero. Furthermore, the following theorem guarantees that there is always at least one vertex with degree one:

Theorem Consider a connected, undirected graph [image], where [image]. Let [image] be a spanning tree of G. The spanning tree T contains at least one vertex of degree one.

extbfProof (By contradiction). Assume that there is no vertex in T of degree one. I.e., all the vertices in T have degree two or greater. Then by following edges into and out of vertices we can construct a path that is cyclic. But a spanning tree is acyclic--a contradiction. Therefore, a spanning tree always contains at least one vertex of degree one.

According to Definition gif , the edge set of a spanning tree is a subset of the edges in the spanned graph. How many edges must a spanning tree have? The following theorem answers the question:

Theorem Consider a connected, undirected graph [image]. Let [image] be a spanning tree of G. The number of edges in the spanning tree is given by

extbfProof (By induction). We can prove Theorem gif by induction on [image], the number of vertices in the graph.

Base Case Consider a graph that contains only one node, i.e., [image]. Clearly, the spanning tree for such a graph contains no edges. Since [image], the theorem is valid.

Inductive Hypothesis Assume that the number of edges in a spanning tree for a graph with [image] has been shown to be [image] for [image].

Consider a graph [image] with k+1 vertices and its spanning tree [image]. According to Theorem gif , [image] contains at least one vertex of degree one. Let [image] be one such vertex and [image] be the one edge emanating from v in [image].

Let [image] be the graph of k nodes obtained by removing v and its emanating edge from the graph [image]. I.e., [image].

Since [image] is connected, so too is [image]. Similarly, since [image] is acyclic, so too is [image]. Therefore [image] is a spanning tree with k vertices. By the inductive hypothesis [image] has k-1 edges. Thus, [image] as k edges.

Therefore, by induction on k, the spanning tree for a graph with [image] vertices contains [image] edges.

  • Constructing Spanning Trees
  • Minimum-Cost Spanning Trees
  • Prim's Algorithm
  • Kruskal's Algorithm


Constructing Spanning Trees

Any traversal of a connected, undirected tree visits all the vertices in that tree, regardless of the node from which the traversal is started. During the traversal certain edges are traversed while the remaining edges are not. Specifically, an edge is traversed if it leads from a vertex that has been visited to a vertex that has not been visited. The set of edges which are traversed during a traversal forms a spanning tree.

The spanning tree obtained from a breadth-first traversal starting at vertex v of graph G is called the breadth-first spanning tree of G rooted at v. For example, the spanning tree shown in Figure gif (c) is the breadth-first spanning tree of [image] rooted at vertex b.

Similarly, the spanning tree obtained from a depth-first traversal is the depth-first spanning tree of G rooted at v. The spanning tree shown in Figure gif (d) is the depth-first spanning tree of [image] rooted at vertex c.


Minimum-Cost Spanning Trees

The total cost of an edge-weighted undirected graph is simply the sum of the weights on all the edges in that graph. A minimum-cost spanning tree of a graph is a spanning tree of that graph that has the least total cost:

Definition (Minimal Spanning Tree) Consider an edge-weighted, undirected, connected graph [image], where C(v,w) represents the weight on edge [image]. The minimum spanning tree of G is the spanning tree [image] that has the smallest total cost,

Figure gif shows edge-weighted graph [image] together with its minimum-cost spanning tree [image]. In general, it is possible for a graph to have several different minimum-cost spanning trees. However, in this case there is only one.

figure52621
Figure: An Edge-Weighted, Undirected Graph and a Minimum-Cost Spanning Tree

The two sections that follow present two different algorithms for finding the minimum-cost spanning tree. Both algorithms are similar in that they build the tree one edge at a time.


Prim's Algorithm

Prim's algorithm finds a minimum-cost spanning tree of an edge-weighted, connected, undirected graph [image]. The algorithm constructs the minimum-cost spanning tree of a graph by selecting edges from the graph one-by-one and adding those edges to the spanning tree.

Prim's algorithm is essentially a minor variation of Dijkstra's algorithm (see Section gif ). To construct the spanning tree, the algorithm constructs a sequence of spanning trees [image], each of which is a subgraph of G. The algorithm begins with a tree that contains one selected vertex, say [image]. I.e., [image].

Given [image], we obtain the next tree in the sequence as follows. Consider the set of edges given by

The set [image] contains all the edges [image] such that exactly one of v or w is in [image] (but not both). Select the edge [image] with the smallest edge weight,

Then [image], where [image] and [image]. After [image] such steps we get [image] which is the minimum-cost spanning tree of G.

Figure gif illustrates how Prim's algorithm determines the minimum-cost spanning tree of the graph [image] shown in Figure gif . The circled vertices are the elements of [image], the solid edges represent the elements of [image] and the dashed edges represent the elements of [image].

figure52993
Figure: Operation of Prim's Algorithm

  • Implementation


Implementation

An implementation of Prim's algorithm is shown in Program gif . This implementation is almost identical to the version of Dijkstra's algorithm given in Program gif . In fact, there are only four differences between the two algorithms. These are found on lines 1, 23, 38, and 40-41.

program53687
Program: Prim's Algorithm

The PrimsAlgorithm function takes two arguments. The first is a const reference to a undirected graph instance. We assume that the graph is edge-weighted and that the weights are instances of the Int class defined in Program gif . The second argument is a const reference to the start node.

The PrimsAlgorithm routine returns a minimum-cost spanning tree represented as an undirected graph. Therefore, the return value is a reference to a Graph instance. The function allocates the storage, constructs the spanning tree and returns a reference to that graph.

The running time of Prim's algorithm is asymptotically the same as Dijkstra's algorithm. I.e., the worst-case running time is

when adjacency lists are used, and

when adjacency matrices are used to represent the input graph.


Kruskal's Algorithm

Like Prim's algorithm, Kruskal's algorithm also constructs the minimum spanning tree of a graph by adding edges to the spanning tree one-by-one. At all points during its execution the set of edges selected by Prim's algorithm forms exactly one tree. On the other hand, the set of edges selected by Kruskal's algorithm forms a forest of trees.

Kruskal's algorithm is conceptually quite simple. The edges are selected and added to the spanning tree in increasing order of their weights. An edge is added to the tree only if it does not create a cycle.

The beauty of Kruskal's algorithm is the way that potential cycles are detected. Consider an undirected graph [image]. We can view the set of vertices, [image], as a universal set and the set of edges, [image], as the definition of an equivalence relation over the universe [image]. (See Definition gif ). In general, an equivalence relation partitions a universal set into a set of equivalence classes. If the graph is connected, there is only one equivalence class--all the elements of the universal set are equivalent. Therefore, a spanning tree is a minimal set of equivalences that result in a single equivalence class.

Kruskal's algorithm computes, [image], a sequence of partitions of the set of vertices [image]. (Partitions are discussed in Section gif ). The initial partition consists of [image] sets of size one:

Each subsequent element of the sequence is obtained from its predecessor by joining two of the elements of the partition. Therefore, [image] has the form

for [image].

To construct the sequence the edges in [image] are considered one-by-one in increasing order of their weights. Suppose we have computed the sequence up to [image] and the next edge to be considered is [image]. If v and w are both members of the same element of partition [image], then the edge forms a cycle, and is not part of the minimum-cost spanning tree.

On the other hand, suppose v and w are members of two different elements of partition [image], say [image] and [image] (respectively). Then [image] must be an edge in the minimum-cost spanning tree. In this case, we compute [image] by joining [image] and [image]. I.e., we replace [image] and [image] in [image] by the union [image].

Figure gif illustrates how Kruskal's algorithm determines the minimum-cost spanning tree of the graph [image] shown in Figure gif . The algorithm computes the following sequence of partitions:

figure53726
Figure: Operation of Kruskal's Algorithm

  • Implementation
  • Running Time Analysis


A Simple Example

Consider the function f(n)=8n+128 shown in Figure gif . Clearly, f(n) is non-negative for all integers [image]. We wish to show that [image]. According to Definition gif , in order to show this we need to find an integer [image] and a constant c>0 such that for all integers [image], [image].

It does not matter what the particular constants are--as long as they exist! E.g., suppose we choose c=1. Then

Since (n+8)>0 for all values of [image], we conclude that [image]. I.e., [image].

So, we have that for c=1 and [image], [image] for all integers [image]. Hence, [image]. Figure gif clearly shows that the function [image] is greater than the function f(n)=8n+128 to the right of n=16.

Of course, there are many other values of c and [image] that will do. For example, c=2 and [image] will do, as will c=4 and [image]. (See Figure gif ).

figure1337
Figure: Showing that [image].


Implementation

An implementation of Kruskal's algorithm is shown in Program gif . The KruskalsAlgorithm function takes as its lone argument a const reference to an edge-weighted, undirected graph. This implementation assumes that the edge weights are instances of the Int class defined in Program gif . The routine computes the minimum-cost spanning tree and returns it in the form of an edge-weighted undirected graph. The function allocates the storage, constructs the graph and returns a reference to a Graph instance.

program54744
Program: Kruskal's Algorithm

The main data structures used by the routine are a priority queue to hold the edges, a partition to detect cycles and a graph for the result. This implementation uses a BinaryHeap (Section gif ) for the priority queue, a PartitionAsForest (Section gif ) for the partition and a GraphAsLists for the spanning tree.


Running Time Analysis

The KruskalsAlgorithm routine begins by creating an graph to hold the result spanning tree (lines 3-7). Since a spanning tree is a sparse graph the GraphAsLists class is used to represent it. Initially the graph contains [image] vertices but no edges. The running time for lines 3-7 is [image].

Next all of the edges in the input graph are inserted one-by-one into the priority queue (lines 9-18). Since there are [image] edges, the worst-case running time for a single insertion is [image]. Therefore, the worst-case running time to initialize the priority queue is

when adjacency lists are used, and

when adjacency matrices are used to represent the input graph.

The main loop of the routine comprises lines 21-36. This loop is done at most [image] times. In each iteration of the loop, one edge is removed from the priority queue (lines 23-25). In the worst-case this takes [image] time.

Then, two partition find operations are done to determine the elements of the partition that contain the two end-points of the given edge (lines 26-29). Since the partition contains at most [image] elements, the running time for the find operations is [image]. If the two elements of the partition are distinct, then an edge is added to the spanning tree and a join operation is done to unite the two elements of the partition (lines 30-34). The join operation also requires [image] time in the worst-case. Therefore, the total running time for the main loop is [image].

Thus, the worst-case running time for Kruskal's algorithm is

when adjacency lists are used, and

when adjacency matrices are used to represent the input graph.


Application: Critical Path Analysis

In the introduction to this chapter it is stated that there are myriad applications of graphs. In this section we consider one such application--critical path analysis . Critical path analysis crops up in a number of different contexts, from the planning of construction projects to the analysis of combinational logic circuits.

For example, consider the scheduling of activities required to construct a building. Before the foundation can be poured, it is necessary to dig a hole in the ground. After the building has been framed, the electricians and the plumbers can rough-in the electrical and water services and this rough-in must be completed before the insulation is put up and the walls are closed in.

We can represent the set of activities and the scheduling constraints using a vertex-weighted, directed acyclic graph (DAG). Each vertex represents an activity and the weight on the vertex represents the time required to complete the activity. The directed edges represent the sequencing constraints. I.e., an edge from vertex v to vertex w indicates that activity v must complete before w may begin. Clearly, such a graph must be acyclic.

A graph in which the vertices represent activities is called an activity-node graph . Figure gif shows an example of of an activity-node graph. In such a graph it is understood that independent activities may proceed in parallel. For example, after activity A is completed, activities B and C may proceed in parallel. However, activity D cannot begin until both B and C are done.

figure54769
Figure: An Activity-Node Graph

Critical path analysis answers the following questions:

  1. What is the minimum amount of time needed to complete all activities?
  2. For a given activity v, is it possible to delay the completion of that activity without affecting the overall completion time? If yes, by how much can the completion of activity v be delayed?

The activity-node graph is a vertex-weighted graph. However, the algorithms presented in the preceding sections all require edge-weighted graphs. Therefore, we must convert the vertex-weighted graph into its edge-weighted dual . In the dual graph the edges represent the activities, and the vertices represent the commencement and termination of activities. For this reason, the dual graph is called an event-node graph .

Figure gif shows the event-node graph corresponding to the activity node graph given in Figure gif . Where an activity depends on more than one predecessor it is necessary to insert dummy edges.

figure55363
Figure: The Event-Node Graph corresponding to Figure gif

For example, activity D cannot commence until both B and C are finished. In the event-node graph vertex 2 represents the termination of activity B and vertex 3 represents the termination of activity C. It is necessary to introduce vertex 4 to represent the event that both B and C have completed. Edges [image] and [image] represent this synchronization constraint. Since these edges do not represent activities, the edge weights are zero.

For each vertex v in the event node graph we define two times. The first [image] is the earliest event time for event v. It is the earliest time at which event v can occur assuming the first event begins at time zero. The earliest event time is given by

where [image] is the initial event, [image] is the set of incident edges on vertex w and C(v,w) is the weight on vertex (v,w).

Similarly, [image] is the latest event time for event v. It is the latest time at which event v can occur The latest event time is given by

where [image] is the final event.

Given the earliest and latest event times for all events, we can compute time available for each activity. E.g., consider an activity represented by edge (v,w). The amount of time available for the activity is [image] and the time required for that activity is C(v,w). We define the slack time for an activity as the amount of time by which an activity can be delayed with affecting the overall completion time of the project. The slack time for the activity represented by edge (v,w) is given by

Activities with zero slack are critical . I.e., critical activities must be completed on time--any delay affects the overall completion time. A critical path is a path in the event-node graph from the initial vertex to the final vertex comprised solely of critical activities.

Table gif gives the results from obtained from the critical path analysis of the activity-node graph shown in Figure gif . The tabulated results indicate the critical path is

Table: Critical Path Analysis Results for the Activity-Node Graph in Figure gif
activity | C(v,w) |  |  | S(v,w)
A | 3 | 0 | 3 | 0
B | 1 | 3 | 7 | 3
C | 4 | 3 | 7 | 0
D | 1 | 7 | 8 | 0
E | 9 | 8 | 17 | 0
F | 5 | 8 | 17 | 4
G | 2 | 17 | 18 | 0

  • Implementation


Implementation

Given an activity-node graph, the objective of critical path analysis is to determine the slack time for each activity and thereby to identify the critical activities and the critical path. We shall assume that the activity node graph has already been transformed to an edge-node graph. The implementation of this transformation is left as a project for the reader (Project gif ). Therefore, the first step is to compute the earliest and latest event times.

According to Equation gif , the earliest event time of vertex w is obtained from the earliest event times of all its predecessors. Therefore, must compute the earliest event times in topological order. To do this, we define the EarliestTimeVisitor shown in Program gif .

program55632
Program: Critical Path Analysis--Computing Earliest Event Times

The EarliestTimeVisitor has three member variables--graph, earliestTime and startTime. The first is a reference to the event-node graph; the second refers to an array used to record the [image] values; and the third is the time at which the initial activity starts.

The Visit member function of the EarliestTimeVisitor class implements directly Equation gif . It uses an IncidentEdges iterator to determine all the predecessors of a given node and computes [image].

In order to compute the latest event times, it is necessary to define also a LatestTimeVisitor. This visitor must visit the vertices of the event-node graph in reverse topological order. Its implementation follows directly from Equation gif and Program gif .

Program gif defines the routine called CriticalPathAnalysis that does what its name implies. This routine takes as its lone argument a reference to a Digraph instance that represents an event-node graph. This implementation assumes that the edge weights are instances of the Int class defined in Program gif .

program55661
Program: Critical Path Analysis--Finding the Critical Paths

The routine first uses the EarliestTimeVisitor in a topological order traversal to compute the earliest event times which are recored in the earliestTime array (lines 5-8). Next, the latest event times are computed and recorded in the latestTime array. Notice that this is done using a LatestTimeVisitor in a postorder depth-first traversal (lines 10-13). This is because a postorder depth-first traversal is equivalent to a topological order traversal in reverse!

Once the earliest and latest event times have been found, we can compute the slack time for each edge. In the implementation shown, an edge-weighted graph is constructed that is isomorphic with the the original event-node graph, but in which the edge weights are the slack times as given by Equation gif (lines 15-31). By constructing such a graph we can make use of Dijkstra's algorithm find the shortest path from start to finish since the shortest path must be the critical path (line 34).

The DijkstrasAlgorithm routine given in Section gif returns its result in the form of a shortest-path graph. The shortest-path graph for the activity-node graph of Figure gif is shown in Figure gif . By following the path in this graph from vertex 9 back to vertex 0, we find that the critical path is [image].

figure55678
Figure: The Critical Path Graph corresponding to Figure gif


Exercises

  1. Consider the undirected graph [image] shown in Figure gif . List the elements of [image] and [image]. Then, for each vertex [image] do the following:
    1. Compute the in-degree of v.
    2. Compute the out-degree of v.
    3. List the elements of [image].
    4. List the elements of [image].

    figure55980
    Figure: Sample Graphs

  2. Consider the directed graph [image] shown in Figure gif .
    1. Show how the graph is represented using an adjacency matrix.
    2. Show how the graph is represented using adjacency lists.
  3. Repeat Exercises gif and gif for the directed graph [image] shown in Figure gif .
  4. Consider a depth-first traversal of the undirected graph [image] shown in Figure gif starting from vertex a.
    1. List the order in which the nodes are visited in a preorder traversal.
    2. List the order in which the nodes are visited in a postorder traversal.
    Repeat this exercise for a depth-first traversal starting from vertex d.
  5. List the order in which the nodes of the undirected graph [image] shown in Figure gif are visited by a breadth-first traversal that starts from vertex a. Repeat this exercise for a breadth-first traversal starting from vertex d.
  6. Repeat Exercises gif and gif for the directed graph [image] shown in Figure gif .
  7. List the order in which the nodes of the directed graph [image] shown in Figure gif are visited by a topological order traversal that starts from vertex a.
  8. Consider an undirected graph [image]. If we use a [image] adjacency matrix A to represent the graph, we end up using twice as much space as we need because A contains redundant information. I.e., A is symmetric about the diagonal and all the diagonal entries are zero. Show how a one-dimensional array of length [image] can be used to represent G. Hint: consider just the part of A above the diagonal.
  9. What is the relationship between the sum of the degrees of the vertices of a graph and the number of edges in the graph.
  10. A graph with the maximum number of edges is called a fully connected graph . Draw fully connected, undirected graphs that contain 2, 3, 4 and 5 vertices.
  11. Prove that an undirected graph with n vertices contains at most n(n-1)/2 edges.
  12. Every tree is a directed, acyclic graph (DAG), but there exist DAGs that are not trees.
    1. How can we tell whether a given DAG is a tree?
    2. Devise an algorithm to test whether a given DAG is a tree.
  13. Consider an acyclic, connected, undirected graph G that has n vertices. How many edges does G have?
  14. In general, an undirected graph contains one or more connected components . A connected component of a graph G is a subgraph of G that is connected and contains the largest possible number of vertices. Each vertex of G is a member of exactly one connected component of G.
    1. Devise an algorithm to count the number of connected components in a graph.
    2. Devise an algorithm that labels the vertices of a graph in such a way that all the vertices in a given connected component get the same label and vertices in different connected components get different labels.
  15. A source in an directed graph is a vertex with zero in-degree. Prove that every DAG has at least one source.
  16. What kind of DAG has a unique topological sort?
  17. Under what conditions does a postorder depth-first traversal of a DAG visit the vertices in reverse topological order.
  18. Consider a pair of vertices, v and w, in a directed graph. Vertex w is said to be reachable from vertex v if there exists a path in G from v to w. Devise an algorithm that takes as input a graph, [image], and a pair of vertices, [image], and determines whether w is reachable from v.
  19. An Eulerian walk is a path in an undirected graph that starts and ends at the same vertex and traverses every edge in the graph. Prove that in order for such a path to exist, all the nodes must have even degree.
  20. Consider the binary relation [image] defined for the elements of the set [image] as follows:

    How can we determine whether [image] is a total order?

  21. Show how the single-source shortest path problem can be solved on a DAG using a topological-order traversal. What is the running time of your algorithm?
  22. Consider the directed graph [image] shown in Figure gif . Trace the execution of Dijkstra's algorithm as it solves the single-source shortest path problem starting from vertex a. Give your answer in a form similar to Table gif .

    figure56311
    Figure: Sample Weighted Graphs

  23. Dijkstra's algorithm works as long as there are no negative edge weights. Given a graph that contains negative edge weights, we might be tempted to eliminate the negative weights by adding a constant weight to all of the edge weights to make them all positive. Explain why this does not work.
  24. Dijkstra's algorithm can be modified to deal with negative edge weights (but not negative cost cycles) by eliminating the known flag [image] and by inserting a vertex back into the queue every time its tentative distance [image] decreases. Explain why the modified algorithm works correctly. What is the running time of the modified algorithm?
  25. Consider the directed graph [image] shown in Figure gif . Trace the execution of Floyd's algorithm as it solves the all-pairs shortest path problem.
  26. Prove that if the edge weights on an undirected graph are distinct, there is only one minimum-cost spanning tree.
  27. Consider the undirected graph [image] shown in Figure gif . Trace the execution of Prim's algorithm as it finds the minimum-cost spanning tree starting from vertex a.
  28. Repeat Exercise gif using Kruskal's algorithm.
  29. Do Exercise gif .


Projects

  1. Devise a graph description language. Implement a routine that reads the description of a graph and constructs a graph object instance. Your routine should be completely generic--it should not depend on the graph implementation used.
  2. Extend Project gif by writing a routine that prints the description of a given graph object instance.
  3. Complete the implementation of the GraphAsMatrix class declared in Program gif by providing suitable definitions for the following member functions: GraphAsMatrix (constructor), GraphAsMatrix (destructor), Purge, AddVertex, SelectVertex, AddEdge, SelectEdge, IsEdge, Vertices, Edges, IncidentEdges and EmanatingEdges. Write a test program and test your implementation.
  4. Repeat Project gif for the GraphAsLists class.
  5. The DigraphAsMatrix class can be implemented using multiple inheritance:
    class DigraphAsMatrix : public Digraph, public GraphAsMatrix
{
    ...
};
    Implement the DigraphAsMatrix class by providing suitable definitions for the following member functions: DigraphAsMatrix (constructor), DigraphAsMatrix (destructor), Purge, AddEdge, SelectEdge, IsEdge and Edges. You must also have a complete implementation of the base class GraphAsMatrix (see Project gif ). Write a test program and test your implementation.
  6. Repeat Project gif for the DigraphAsLists class.
  7. Add a member function to the Digraph class abstract interface that creates an instance of and returns a reference to an undirected graph which underlies the given digraph.
  8. Devise an approach using an iterator and a stack to perform a topological-order traversal by doing a postorder depth-first traversal in reverse.
  9. The single-source shortest path problem on a DAG can be solved by visiting the vertices in topological order. Write an visitor for use with the TopologicalOrderTraversal routine that solves the single-source shortest path problem on a DAG.
  10. Devise and implement an routine that transforms a vertex-weighted activity-node graph into an edge-weighted event-node graph.
  11. Complete the implementation of the critical path analysis routines. In particular, you must implement the LatestTimeVisitor along the lines of the EarliestTimeVisitor defined in Program gif .


C++ and Object-Oriented Programming

This appendix is a brief review of the fundamentals of programming in C++. It identifies and describes the features of C++ that are used throughout this text. This appendix is not a C++ tutorial--if you are not familiar with C++, you should read one of the myriad C++ programming books.

  • Variables, Pointers and References
  • Parameter Passing
  • Objects and Classes
  • Inheritance and Polymorphism
  • Templates
  • Exceptions


Variables, Pointers and References

A variable is a programming language abstraction that represents a storage location. A variable has the following attributes:

name
The name of a variable is the label used to identify a variable in the text of a program.
address
The address of a variable is the memory address of the storage location(s) occupied by that variable. The address attribute of a variable is also known as the l-value because the l-value of a variable is required when a variable name is used on the left side of an assignment statement.
size
The size of a variable is the amount of storage (in bytes) occupied by that variable.
type
The type of a variable determines the set of values that the variable can have and the set of operations that can be performed on that variable.
value
The value of a variable is the content of the memory location(s) occupied by that variable. (How the contents of the memory locations are interpreted is determined by the type of the variable). The value attribute of a variable is also known as the r-value because the r-value of a variable is required when a variable name is used on the right side of an assignment statement.
lifetime
The lifetime of a variable is the interval of time in the execution of a program during which a variable is said to exist. Some variables exist for the entire execution of a program; some are created and destroyed automatically during the execution of a program; and others are explicitly created and destroyed by the programmer.
scope
The scope of a variable is the range of statements in the text of a program in which that variable can be referenced.

Consider the C++ variable declaration statement:

int i = 57;
This statement defines a variable and binds various attributes with that variable. The name of the variable is i, the type of the variable is int, its size is [image] (typically two or four bytes), and its initial value is 57.

Some attributes of a variable, such its name, type and size, are bound at compile time. This is called static binding. Other attributes of a variable, such as its address and value, may be bound at run time. This is called dynamic binding.

  • Pointers Are Variables
  • References are Not Variables


Pointers Are Variables

The C++ statement

int* p;
declares a pointer . A pointer is a variable the type of which has the form [image] where T is an arbitrary type. In this case, [image] and we say that p is a pointer to int.

Since a pointer is a variable, it has all the attributes of a variable. In particular, it has both an address (l-value) and a value (r-value).

Consider a C++ program that contains the following global variable declarations:

int i = 57;
int j = 31;
int* p = 0;
int* q = (int*) 1004;
Suppose the four variables, i, j, p and q, are stored in contiguous memory locations starting at address 1000 as shown in Figure gif . (We assume that [image]). Note that the variable named j has the value 31 and is found at address 1000. Similarly, the variable named q has the value 1004 and is found at address 1012.

figure56985
Figure: Memory Layout of C++ Variables and Pointers

An assignment statement such as

i = j;
takes the value (r-value) of j, in this case 31, and stores it in memory at the address (l-value) of i, in this case 1000. Similarly, the assignment
p = q;
takes the value of q, in this case 1004, and stores it in memory at the address of p, in this case 1008.

  • Dereferencing Pointers


Dereferencing Pointers

A variable of type [image] is said to point at an object of type T. What this really means is that the value of the [image] variable is the address of another variable of type T. For example, in Figure gif the variable q is has type int*, i.e., it is a pointer to an int. The value of p is 1004, which is the address of the variable j.

To access the variable to which a pointer points, we must dereference the pointer. E.g., to dereference the pointer q we write *q. Whereas q denotes the pointer variable itself, *q denotes the int to which the pointer points.

If we use *q in a context where an r-value is expected, then we get the r-value of the variable to which q points. E.g., since q points to the variable j, the assignment

i = *q;
takes the r-value of j (31) and assigns it to i.

Similarly, if we use *q in a context where an l-value is required, then we get the l-value of the variable to which q points. E.g., since q points to the variable j, the assignment

*q = 31;
takes the l-value of j (1004) and stores 31 at that location. Notice that the l-value of *q is identical to the r-value of q.


Big Oh Fallacies and Pitfalls

Unfortunately, the way we write big oh notation can be misleading to the naıve reader. This section presents two fallacies which arise because of a misinterpretation of the notation.

Fallacy Given that [image] and [image], then [image].

Consider the equations:

Clearly, it is reasonable to conclude that [image].

However, consider these equations:

It does not follow that [image]. E.g., [image] and [image] are both [image], but they are not equal.

Fallacy If f(n)=O(g(n)), then [image].

Consider functions f, g, and h, such that f(n)=h(g(n)). It is reasonable to conclude that [image] provided that [image] is an invertible function. However, while we may write f(n)=O(h(n)), the equation [image] is nonsensical and meaningless. Big oh is not a mathematical function, so it has no inverse!

The reason for these difficulties is that we should read the notation [image] as ``f(n) is big oh n squared'' not ``f(n) equals big oh of n squared.'' The equal sign in the expression does not really denote mathematical equality! And the use of the functional form, [image], does not really mean that O is a mathematical function!


References are Not Variables

In C++, a reference is an alternative name for a variable. The declaration of a reference looks very much like the declaration of a variable. However, a reference is not a variable. The notation [image] denotes a reference to a variable of type T. For example the code sequence

int i = 57;
int& r = i;
defines one variable, i, and one reference, r.

In C++ all references must be initialized. In this case the reference r refers to the variable i. A reference can be used in a program everywhere that a variable can be used. In particular, if the reference r is used where a r-value is required, it is the r-value of i that is obtained. Similarly, if the reference r is used where an l-value is required, it is the l-value of i that is obtained.

Is it important to note that in C++, despite appearances, there are no operators that operate on references. In particular, it is not possible to change the variable to which a reference refers. And since every reference must be initialized by associating it with a variable, there is no such thing as a null reference. References are mainly used in C++ programs to specify the arguments and return values of functions.


Parameter Passing

Parameter passing methods are the ways in which parameters are transfered between functions when one function calls another. C++ provides two parameter passing methods--pass-by-value and pass-by-reference .

  • Pass By Value
  • Pass By Reference


Pass By Value

Consider a pair of C++ functions defined in Program gif . The function One calls the function Two. In general, every function call includes a (possibly empty) list of arguments. The arguments specified in a function call are called actual parameters . In this case, there is only one actual parameter--y.

program57154
Program: Example of Pass-By-Value Parameter Passing

The method by which the parameter is passed to a function is determined by the function definition. In this case, the function Two is defined as accepting a single argument of type int called x. The arguments which appear in a function definition are called formal parameters . If the type of a formal parameter is not a reference (see Section gif ), then the parameter passing method is pass-by-value.

The semantics of pass-by-value work like this: The effect of the formal parameter definition is to create a local variable of the specified type in the given function. E.g., the function Two has a local variable of type int called x. When the function is called, the values (r-values) of the actual parameters are used to initialize the formal parameters before the body of the function is executed.

Since the formal parameters give rise to local variables, if a new value is assigned to a formal parameter, that value has no effect on the actual parameters. Therefore, the output obtained produced by the function One defined in Program gif is:

2
1


Pass By Reference

Consider the pair of C++ functions defined in Program gif . The only difference between this code and the code given in Program gif is the definition of the formal parameter of the function Two: In this case, the parameter x is declared to be a reference to an int. In general, if the type of a formal parameter is a reference, then the parameter passing method is pass-by-reference .

program57185
Program: Example of Pass-By-Reference Parameter Passing

A reference formal parameter is not a variable. When a function is called that has a reference formal parameter, the effect of the call is to associate the reference with the corresponding actual parameter. I.e., the reference becomes an alternative name for the corresponding actual parameter. Consequently, this means that the actual parameter passed by reference must be variable.

A reference formal parameter can be used in the called function everywhere that a variable can be used. In particular, if the reference formal parameter is used where a r-value is required, it is the r-value of actual parameter that is obtained. Similarly, if the reference parameter is used where an l-value is required, it is the l-value of actual parameter that is obtained. Therefore, the output obtained produced by the function One defined in Program gif is:

2
2

  • The Trade-off
  • Constant Parameters


The Trade-off

Clearly, the parameter passing method used constrains the functionality of the called function: When pass-by-value is used, the called function cannot modify the actual parameters; when pass-by-reference is used, the called function is able to modify the actual parameters. In addition, the two methods have different time and space requirements that need to be understood in order to make the proper selection.

Pass-by-value creates a local variable and initializes that local variable by copying the value of the actual parameter. This means that space is used (on the stack) for the local variable and that time is taken to initialize that local variable. For small variables these penalties are insignificant. However, if the variable is large, the time and space penalties may become prohibitive.

On the other hand, pass-by-reference does not create a local variable nor does it require the copying of the actual parameter. However, because of the way that it must be implemented, every time a reference formal parameter is used to access the corresponding actual parameter, a small amount of extra time is taken to dereference the reference. As a result, it is typically more efficient to pass small variables by value and large variables by reference.


Constant Parameters

In general, pass-by-reference parameter passing allows the called function to modify the actual parameters. However, sometimes it is the case that the programmer does not want the actual parameters to be modified. Nevertheless, pass-by-reference may be the preferred method for performance reasons.

In C++ we can use the const keyword to achieve the performance of pass-by-reference while at the same time ensuring that the actual parameter cannot be modified. Consider the following definition of the function Two:

void Two (int const& x)
{
    x = 2; // Not allowed.
    cout << x << endl; // This is ok.
}
The const keyword modifies the type int. It says that the int to which x refers is a constant. The value of x can be used without impunity. However, x cannot be used as the target of an assignment statement. In fact, the variable x cannot be used in any context where it might be modified. Any attempt to do so is an error that is detected by the compiler.


Objects and Classes

``An object is a region of storage[12].'' The type of the object determines how that region of storage is interpreted and how it can be manipulated. C++ provides built-in types , such as char, int and float, and it supports the creation of user-defined types using the class construct.

The class construct is what makes C++ an object-oriented language. A C++ class definition groups a set of values with a set of operations. Classes facilitate modularity and information hiding. The user of a class manipulates object instances of that class only through a well-defined interface.

It is often the case that different classes possess common features. Different classes may share common values; they may perform the same operations; they may support common interfaces. In C++ such relationships are expressed using derivation and inheritance.

  • Member Variables and Member Functions
  • Constructors and Destructors
  • Accessors and Mutators


Member Variables and Member Functions

A class groups a set of values and a set of operations. The values and the operations of a class are called its members. Member variables implement the values and member functions implement the operations.

Suppose we wish to define a class to represent complex numbers . The Complex class definition shown in Program gif illustrates how this can be done. Two member variables, real and imag, are declared. These represent the real and imaginary parts of a complex numbers (respectively).

program57230
Program: Complex Class Definition

Every object instance of the Complex class contains its own member variables. Consider the following variable declarations:

Complex c;
Complex d;
Both c and d are instances of the complex class. Therefore, each of them has a real and imag member variable. The member variables of an object are accessed using the dot operator. E.g., c.real refers to the real member variable of c and d.imag refers to the imag member of d.

The Complex class definition also contains a list of function prototypes which declare the member functions of the class. In general, a member function performs some operation on an instances of the class. Again, the dot operator is used to specify the object on which the operation is performed. E.g., c.SetReal(1.0) invokes the SetReal function on c and d.Put(cout) invokes the Put function on d.


Constructors and Destructors

A constructor is a member function that has the same name as its class (and which has no return value). E.g., there are four constructors declared in Program gif (lines 6-9). The purpose of a constructor is to initialize an object. In fact, C++ makes the following promise: If a class has a constructor, all objects of that class will be initialized.

Consider the following sequence of variable declarations:

Complex c;       // calls Complex ()
Complex d = 2.0; // calls Complex (double)
Complex i(0, 1); // calls Complex (double, double)
Complex j(i);    // calls Complex (Complex const&)
Each variable declaration implicitly causes the appropriate constructor to be invoked. In particular, the equal sign in the declaration of d denotes initialization not assignment and, therefore, is accomplished by calling the constructor.

  • Default Constructor
  • Copy Constructor
  • The Copy Constructor, Parameter Passing and Function Return Values
  • Destructors


Default Constructor

The constructor which takes no arguments is called the default constructor . E.g., the default constructor is invoked when a variable is declared like this:

Complex c;
In fact, the compiler will generate its own default constructor for a class provided that no other constructors have been defined by the programmer. The compiler generated default constructor initializes the member variables of the class using their respective default constructors.

Program gif gives the implementation of the default constructor of the Complex class. Because the implementation the member function is given outside of the class definition, the name of the function is prefixed with the class name followed by ::.

program57292
Program: Complex Class Constructors and Destructor

The default constructor explicitly invokes the constructors for the member variables real and imag (line 2). After the member variables have been initialized, the body of the constructor runs. In this case, the body of the constructor is empty (line 3).


Outline

This book presents material identified in the Computing Curricula 1991 report of the ACM/IEEE-CS Joint Curriculum Task Force[38]. The book specifically addresses the following knowledge units: AL1: Basic Data structures, AL2: Abstract Data Types, AL3: Recursive Algorithms, AL4: Complexity Analysis, AL6: Sorting and Searching, and AL8: Problem-Solving Strategies. The breadth and depth of coverage is typical of what should appear in the second or third year of an undergraduate program in computer science/computer engineering.

In order to analyze a program, it is necessary to develop a model of the computer. Chapter gif develops several models and illustrates with examples how these models predict performance. Both average-case and worst-case analyses of running time are considered. Recursive algorithms are discussed and it is shown how to solve a recurrence using repeated substitution. This chapter also reviews arithmetic and geometric series summations, Horner's rule and the properties of harmonic numbers.

Chapter gif introduces asymptotic (big-oh) notation and shows by comparing with Chapter gif that the results of asymptotic analysis are consistent with models of higher fidelity. In addition to [image], this chapter also covers other asymptotic notations ( [image], [image] and [image]) and develops the asymptotic properties of polynomials and logarithms.

Chapter gif introduces the foundational data structures--the array and the linked list. Virtually all the data structures in the rest of the book can be implemented using either one of these foundational structures. This chapter also covers multi-dimensional arrays and matrices.

Chapter gif deals with abstraction and data types. It presents the recurring design patterns used throughout the text as well a unifying framework for the data structures presented in the subsequent chapters. In particular, all of the data structures are viewed as abstract containers.

Chapter gif discusses stacks, queues and deques. This chapter presents implementations based on both foundational data structures (arrays and linked lists). Applications for stacks and queues and queues are presented.

Chapter gif covers ordered lists, but sorted and unsorted. In this chapter, a list is viewed as a searchable container. Again several applications of lists are presented.

Chapter gif introduces hashing and the notion of a hash table. This chapter addresses the design of hashing functions for the various basic data types as well as for the abstract data types described in Chapter gif . Both scatter tables and hash tables are covered in depth and analytical performance results are derived.

Chapter gif introduces trees and describes their many forms. Both depth-first and breadth-first tree traversals are presented. Completely generic traversal algorithms based on the use of the visitor design pattern are presented, thereby illustrating the power of algorithmic abstraction. This chapter also shows how trees are used to represent mathematical expressions and illustrates the relationships between traversals and the various expression notations (prefix, infix and postfix).

Chapter gif addresses trees as searchable containers. Again, the power of algorithmic abstraction is demonstrated by showing the relationships between simple algorithms and balancing algorithms. This chapter also presents average case performance analyses and illustrates the solution of recurrences by telescoping.

Chapter gif presents several priority queue implementations, including binary heaps, leftist heaps and binomial queues. In particular this chapter illustrates how a more complicated data structure (leftist heap) extends an existing one (tree). Discrete-event simulation is presented as an application of priority queues.

Chapter gif covers sets and multisets. Also covered are partitions and disjoint set algorithms. The latter topic illustrates again the use of algorithmic abstraction.

Techniques for dynamic storage management are presented in Chapter gif . This is a topic that is not found often in texts of this sort. However, the features of C++ which allow the user to redefine the new and delete operators make this topic approachable.

Chapter gif surveys a number of algorithm design techniques. Included are brute-force and greedy algorithms, backtracking algorithms (including branch-and-bound), divide-and-conquer algorithms and dynamic programming. An object-oriented approach based on the notion of an abstract solution space and an abstract solver unifies much of the discussion. This chapter also covers briefly random number generators, Monte Carlo methods, and simulated annealing.

Chapter gif covers the major sorting algorithms in an object-oriented style based on the notion of an abstract sorter. Using the abstract sorter illustrates the relationships between the various classes of sorting algorithm and demonstrates the use of algorithmic abstractions.

Finally, Chapter gif presents an overview of graphs and graph algorithms. Both depth-first and breadth-first graph traversals are presented. Topological sort is viewed as yet another special kind of traversal. Generic traversal algorithms based on the visitor design pattern are presented, once more illustrating algorithmic abstraction. This chapter also covers various shortest path algorithms and minimum-spanning-tree algorithms.

At the end of each chapter is a set of exercises and a set of programming projects. The exercises are designed to consolidate the concepts presented in the text. The programming projects generally require the student to extend the implementation given in the text.


Properties of Big Oh

In this section we examine some of the mathematical properties of big oh. In particular, suppose we know that [image] and [image].

  • What can we say about the asymptotic behavior of the sum of [image] and [image]? (Theorems gif and gif ).
  • What can we say about the asymptotic behavior of the product of [image] and [image]? (Theorems gif and gif ).
  • How are [image] and [image] related when [image]? (Theorem gif ).

The first theorem addresses the asymptotic behavior of the sum of two functions whose asymptotic behaviors are known:

Theorem If [image] and [image], then

extbfProof By Definition gif , there exist two integers, [image] and [image] and two constants [image] and [image] such that [image] for [image] and [image] for [image].

Let [image] and [image]. Consider the sum [image] for [image]:

Thus, [image].

According to Theorem gif , if we know that functions [image] and [image] are [image] and [image], respectively, the sum [image] is [image]. The meaning of [image] is this context is the function h(n) where [image] for integers all [image].

For example, consider the functions [image] and [image]. Then

Theorem gif helps us simplify the asymptotic analysis of the sum of functions by allowing us to drop the [image] required by Theorem gif in certain circumstances:

Theorem If [image] in which [image] and [image] are both non-negative for all integers [image] such that [image] for some limit [image], then [image].

extbfProof According to the definition of limits , the notation

means that, given any arbitrary positive value [image], it is possible to find a value [image] such that for all

Thus, if we chose a particular value, say [image], then there exists a corresponding [image] such that

Consider the sum [image]:

where [image]. Thus, [image].

Consider a pair of functions [image] and [image], which are known to be [image] and [image], respectively. According to Theorem gif , the sum [image] is [image]. However, Theorem gif says that, if [image] exists, then the sum f(n) is simply [image] which, by the transitive property (see Theorem gif below), is [image].

In other words, if the ratio [image] asymptotically approaches a constant as n gets large, we can say that [image] is [image], which is often a lot simpler than [image].

Theorem gif is particularly useful result. Consider [image] and [image].

From this we can conclude that [image]. Thus, Theorem gif suggests that the sum of a series of powers of n is [image], where m is the largest power of n in the summation. We will confirm this result in Section gif below.

The next theorem addresses the asymptotic behavior of the product of two functions whose asymptotic behaviors are known:

Theorem If [image] and [image], then

extbfProof By Definition gif , there exist two integers, [image] and [image] and two constants [image] and [image] such that [image] for [image] and [image] for [image]. Furthermore, by Definition gif , [image] and [image] are both non-negative for all integers [image].

Let [image] and [image]. Consider the product [image] for [image]:

Thus, [image].

Theorem gif describes a simple, but extremely useful property of big oh. Consider the functions [image] and [image]. By Theorem gif , the asymptotic behavior of the product [image] is [image]. I.e., we are able to determine the asymptotic behavior of the product without having to go through the gory details of calculating that [image].

The next theorem is closely related to the preceding one in that it also shows how big oh behaves with respect to multiplication.

Theorem If [image] and [image] is a function whose value is non-negative for integers [image], then

extbfProof By Definition gif , there exist integers [image] and constant [image] such that [image] for [image]. Since [image] is never negative,

Thus, [image].

Theorem gif applies when we multiply a function, [image], whose asymptotic behavior is known to be [image], by another function [image]. The asymptotic behavior of the result is simply [image].

One way to interpret Theorem gif is that it allows us to do the following mathematical manipulation:

I.e., Fallacy gif notwithstanding, we can multiply both sides of the ``equation'' by [image] and the ``equality'' still holds. Furthermore, when we multiply [image] by [image], we simply bring the [image] inside the [image].

The last theorem in this section introduces the transitive property of big oh:

Theorem (Transitive Property) If f(n)=O(g(n)) and g(n)=O(h(n)) then f(n)=O(h(n)).

extbfProof By Definition gif , there exist two integers, [image] and [image] and two constants [image] and [image] such that [image] for [image] and [image] for [image].

Let [image] and [image]. Then

Thus, f(n)=O(h(n)).

The transitive property of big oh is useful in conjunction with Theorem gif . Consider [image] which is clearly [image]. If we add to [image] the function [image], then by Theorem gif , the sum [image] is [image] because [image]. I.e., [image]. The combination of the fact that [image] and the transitive property of big oh, allows us to conclude that the sum is [image].


Copy Constructor

The copy constructor for class T is a member function of the form T(T&) or T(T const&). The purpose of a copy constructor is to initialize a new object of a given class by copying an existing object of that class. Consider this sequence of declarations:

Complex c;
Complex d(c);
The variable c is initialized using the default constructor, whereas d is initialized by calling the copy constructor. In effect, the value of c is copied into d.

The compiler will generate its own copy constructor for any class in which no copy constructor has been defined by the programmer. The compiler generated copy constructor initializes the member variables of the class using their respective copy constructors.


The Copy Constructor, Parameter Passing and Function Return Values

The copy constructor plays a special rôle in pass-by-value parameter passing. As discussed in Section gif , the semantics of pass-by-value stipulate that the the values of the actual parameters are used to initialize the formal parameters when a function is called. The compiler achieves this by using the copy constructor to copy the value of the actual parameter to the formal parameter.

The copy constructor plays a similar rôle when a value is returned from a function using the return statement. The copy constructor is used to copy the value from the called function back to the point where the calling function makes use of the result.

Program gif gives an implementation for the copy constructor of the Complex class. The copy constructor initializes the the member variables real and imag by calling explicitly their respective copy constructors (line 14). After the member variables have been initialized, the body of the constructor runs. In this case, the body of the copy constructor is empty (line 15).


Destructors

A destructor is a special member function used to finalize an object. The name the destructor for class T is T() and, like constructors, it has no return value. The destructor takes no arguments and, therefore, there is only one destructor per class.

It is normally not necessary to call a destructor explicitly. The destructor for an object is automatically invoked at the end of the lifetime of that object. In particular, this means that the destructors for global variables run after the main routine exits, and that the destructors for local and temporary variables run when the program exits the scope containing those variables.

If the programmer does not define a destructor explicitly, the C++ compiler generates one. The compiler-generated destructor simply invokes the destructors for the member variables of the class. The members of a class are always destroyed in the reverse of the order in which they were constructed.

Program gif defines explicitly a destructor for the class Complex (lines 17-18). In this case, the body of the destructor is empty. When the destructor runs, the body of the destructor is run first. After the body of the destructor has completed, the destructors for the member variables run (automatically).


Accessors and Mutators

An accessor is a member function that accesses the contents of an object but does not modify that object. In the simplest case, an accessor just returns the value of one of the member variables. In general, an accessor performs some computation using the member variables as long as that computation does not modify any of the member variables.

Program gif defines the five accessor member functions of the Complex class--Real, Imag, R, Theta and Put. The fact that these functions are accessors is indicated by the const keyword at the end of the function declarator. The const signifies that this member function is not permitted to modify the object on which it operates. In fact, the C++ compiler will not allow any operations that could modify any of the member variables of the object.

program57344
Program: Complex Class Accessors

By defining suitable accessors, it is possible to hide the implementation of the class from the user of that class. Consider the following statements:

cout << c.real << endl;
cout << c.Real() << endl;
The first statement depends on the implementation of the Complex class. If we change the implementation of the class from the one given (which uses rectangular coordinates) to one that uses polar coordinates, then the first statement above must also be changed. On the other hand, the second statement does not need to be modified, provided we reimplement the Real member function when we switch to polar coordinates.

  • Mutators
  • Member Access Control


Mutators

A mutator is a member function that can modify an object. Every non-const member function of a class is a mutator. In the simplest case, a mutator just assigns a new value to one of the member variables. In general, a mutator performs some computation and modifies any number of member variables.

Program gif defines three mutators for the Complex class. The first two, SetReal and SetImag, simply assign new values to the real and imag member variables, respectively.

program57372
Program: Complex Class Mutators

Program gif also shows how the assignment operator, operator=, can be overloaded . By overloading the assignment operator in this way the function is called whenever a complex number is assigned to another like this:

c = d;

If the programmer does not define an assignment operator explicitly, the C++ compiler generates one. The compiler-generated assignment operator simply assigns one-by-one the member variables of the class by invoking their respective assignment operators.


Member Access Control

Every member of a class, be it a variable or a function, has an access control attribute which affects the manner in which that member can be accessed. The members of a class can be private, public or protected.

By default, the members of a class are private. For example, the member variables real and imag declared in Program gif are both private. Private members can be used only by member functions and friends of the class in which the member is declared.

On the other hand, public members of a class can be used by any function. In Program gif the keyword public on line 5 indicates that all the following member functions are publicly accessible.

In effect, the public part of a class defines the interface to that class and the private part of the class encapsulates the implementation of that class. By making the implementation of a class private, we ensure that the code which uses the class depends only on the interface and not on the implementation of the class. Furthermore, we can modify the implementation of the class without affecting the code of the user of that class.

Protected members are similar to private members. I.e., they can be used by member functions and friends of the class in which the member is declared. In addition, protected members can also be used by member functions and friends of all the classes derived from the class in which the member is declared. The protected category is discussed again in Section gif .


Inheritance and Polymorphism

  • Derivation and Inheritance
  • Polymorphism
  • Multiple Inheritance
  • Run-Time Type Information and Casts


Derivation and Inheritance

This section reviews the concept of a derived class. Derived classes are an extremely useful feature of C++ because they allow the programmer to define new classes by extending existing classes. By using derived classes, the programmer can exploit the commonalities that exist among the classes in a program. Different classes can share values, operations, and interfaces.

Derivation is the definition of a new class by extending one or more existing classes. The new class is called the derived class and the existing classes from which it is derived are called the base classes . Usually there is only one base class. However, C++ allows there to be more than one base class (multiple inheritance ).

Consider the classes Person and Parent defined in Program gif . Because parents are people too, the Parent class is derived from the Person class.

program57424
Program: Derivation and Inheritance in C++

A derived class inherits all the members of all its base classes. I.e., the derived class contains all the member variables contained in the base classes and the derived class supports all the same operations provided by the base classes. E.g., consider the following variable declarations:

Person p;
Parent q;
Since p is a Person, it has the member variables name and sex and member function Name. Furthermore, since Parent is derived from Person, then the object q also has the member variables name and sex and member function Name.

A derived class can extends the base class(es) in several ways: New member variables can be defined, new member functions can be defined, and existing member functions can be overridden . E.g., the Parent class adds the member variables numberOfChildren and child and the member function Child.

If a function is defined in a derived class that has exactly the same signature (name and types of arguments) as a function in a base class, the function in the derived class overrides the one in the base class. E.g., the Put function in the Parent class overrides the Put function in the Person class. Therefore, p.Put(...) invokes Person::Put, whereas q.Put(...) invokes Parent::Put.

An instance of a derived class can be used anywhere in a program where an instance of the base class may be used. For example, this means that a Parent may be passed as an actual parameter to a function (either by value or by reference) in which the formal parameter is a Person.

It is also possible to assign the address of a derived class object to a pointer to a base class object like this:

Person* ptr = new Parent;
However, having done so, it is not possible to call ptr->Child(...), because ptr is a pointer to a Person instance and a Person is not necessarily a Parent.

Similarly, we can use a reference to a Person to refer to a parent:

Person& p = *new Parent;
For the same reasons as above, it is not possible to use the reference p to invoke p.Child(...).

  • Derivation and Access Control


Derivation and Access Control

Members of a class can be private, public or protected. As explained in Section gif , private members are accessible only by member functions and friends of the class in which the member is declared. In particular, this means that the member functions of a derived class cannot access the private members of the base classes even though the derived class has inherited those members! On the other hand, if we make the members of the base classes public, then all classes can access those members directly, not just derived classes.

C++ provides a third category of access control--protected. Protected members can be used by member functions and friends of the class in which the member is declared as well as by member functions and friends of all the classes derived from the class in which the member is declared.

A derived class inherits all the members of its base class(es). The accessibility of the members inherited by a derived class depends on whether the derivation is private or public:

class B { ... };
class D1 : public B { ... };
class D2 : private B { ... };
Here B is a public base class of D1. This means that the public members of B are also public in D1 and that the protected members of B are also protected in D1. On the other hand, B is a private base class of D2. This means that the public and protected members of B are private in D2.


Polymorphism

Polymorphism literally means ``having many forms.'' Polymorphism arises when a set of distinct classes share a common interface because they are all derived from the same base class(es). Because the derived classes are distinct, their implementations may differ. However, because the derived classes share a common interface, instances of those classes are used in exactly the same way.

Consider a program for creating simple drawings. Suppose the program provides a set of primitive graphical objects, such as circles, rectangles and squares. The user of the program selects the desired objects, and then invokes commands to draw, to erase, or to move them about. Ideally, all graphical objects support the same set of operations. Nevertheless, the way that the operations are implemented varies from one object to the next.

We implement this as follows: First, we define a GraphicalObject class which represents the common interface shared by all graphical objects. Then, we derive from GraphicalObject classes to represent circles, rectangles and squares.

Consider the GraphicalObject class declared in Program gif . This class has a single member variable, center, which is an instance of the Point class and which represents the position in the drawing of the center-point of the graphical object. The constructor for the GraphicalObject class takes as its lone argument a reference to a Point and initializes the center member variables accordingly.

program57510
Program: Point and GraphicalObject Class Definitions

In addition to the constructor and destructor, three member functions are declared. The Draw routine draws the object, the Erase routine erase the object, and the MoveTo routine moves the object to a new position.

Program gif shows a possible implementation for the Erase function: In this case we assume that the image is drawn using an imaginary pen. Assuming that we know how to draw a graphical object, we can erase the object by changing the color of the pen so that it matches the background color and then redrawing the object.

program57536
Program: GraphicalObject Class MoveTo and Erase Member Function Definitions

Once we can erase an object as well as draw it, then moving it is easy. Just erase the object, change its center point, and then draw it again. This is how the MoveTo function shown in Program gif is implemented.

We have seen that the GraphicalObject class provides implementations for the Erase and MoveTo member functions. However, the GraphicalObject class does not provide an implementation for the Draw function. Why not? Because until we know what kind of object it is, we cannot possibly know how to draw it!

Consider the Circle class defined in Program gif . This class is derived from GraphicalObject. Therefore, it inherits the member variable center and the member functions Erase and MoveTo. The Circle class adds an additional member variable, radius, and it overrides the Draw routine. The body of the Draw routine is not shown in Program gif . However, we shall assume that it draws a circle with the given radius and center point.

program57567
Program: Circle, Rectangle and Square Class Definitions

Using the Circle class defined in Program gif we can write code like this:

Circle c (Point (0, 0), 5);
c.Draw ();
c.MoveTo (Point (10, 10));
c.Erase ();
This code sequence declares a circle object with its center initially at position (0,0) and radius 5. The circle is then drawn, moved to (10,10), and then erased.

Program gif also defines the classes Rectangle and Square. The Rectangle class is also derived from GraphicalObject. Therefore, it inherits the member variable center and the member functions Erase and MoveTo. The Rectangle class adds two additional member variables, height and width, and it overrides the Draw routine. The body of the Draw routine is not shown in Program gif . However, we shall assume that it draws a rectangle with the given dimensions and center point.

The Square class is derived from the Rectangle class. No new member variables or functions are declared--those inherited from GraphicalObject or from Rectangle are sufficient. The constructor simply arranges to make sure that the height and width of a square are equal!

  • Virtual Member Functions
  • Abstract Classes and Concrete Classes
  • Algorithmic Abstraction


About Polynomials

In this section we examine the asymptotic behavior of polynomials in n. In particular, we will see that as n gets large, the term involving the highest power of n will dominate all the others. Therefore, the asymptotic behavior is determined by that term.

Theorem Consider a polynomial in n of the form

where [image]. Then [image].

extbfProof Each of the terms in the summation is of the form [image]. Since n is non-negative, a particular term will be negative only if [image]. Hence, for each term in the summation, [image]. Recall too that we have stipulated that the coefficient of the largest power of n is positive, i.e., [image].

Note that for integers [image], [image] for [image]. Thus

From Equation gif we see that we have found the constants [image] and [image], such that for all [image], [image]. Thus, [image].

This property of the asymptotic behavior of polynomials is used extensively. In fact, whenever we have a function, which is a polynomial in n, [image] we will immediately ``drop'' the less significant terms (i.e., terms involving powers of n which are less than m), as well as the leading coefficient, [image], to write [image].


Virtual Member Functions

In Program gif Draw, Erase and MoveTo are all declared as virtual member functions of the GraphicalObject class. Declaring a member function virtual changes the way in which the compiler determines the member function to call.

Consider the following sequence of instructions:

GraphicalObject& g1 = *new Circle (Point (0,0), 5);
GraphicalObject& g2 = *new Square (Point (0,0), 5);
g1.Draw ();
g2.Draw ();
If the Draw function was not declared virtual, then both g1.Draw() and g2.Draw() would invoke GraphicalObject::Draw. However, because Draw is a virtual function, g1.Draw() calls Circle::Draw and g2.Draw() calls Rectangle::Draw.

It is as if every object of a class contains a pointer to the actual routine to be invoked when a virtual function is called on that object. E.g, Circle objects carry pointers to Circle::Draw, GraphicalObject::Erase and GraphicalObject::MoveTo, whereas Square objects carry pointers to Rectangle::Draw, GraphicalObject::Erase and GraphicalObject::MoveTo.

Virtual functions ensure that the ``correct'' member function is actually called, regardless of how the object is accessed. Consider the following sequence:

Square s (Point (0,0), 5);
Rectangle& r = s;
GraphicalObject& g = r;
Here s, r and g all refer to the same object, even though they are all of different types. However, because Draw is virtual, s.Draw(), r.Draw() and g.Draw() all invoke Rectangle::Draw.


Abstract Classes and Concrete Classes

In C++ an abstract class is one which defines an interface, but does not necessarily provide implementations for all its member functions. An abstract class is meant to be used as the base class from which other classes are derived. The derived class is expected to provide implementations for the member functions that are not implemented in the base class. A derived class that implements all the missing functionality is called a concrete class .

E.g., the GraphicalObject class defined in Programs gif and gif is an abstract class. In particular, no implementation is given for the virtual member function Draw. The fact that no implementation is given is indicated by the =0 attached to the Draw function prototype in the class definition (Program gif , line 19). Recall that an object carries a pointer to the appropriate routine for every virtual member function it supports. The =0 says that the pointer for the Draw function is not defined in the GraphicalObject class and, therefore, it must be defined in a derived class.

A virtual member function for which no implementation is given is called a pure virtual function . If a C++ class contains a pure virtual function, it is an abstract class. In C++ it is not possible to instantiate an abstract class. E.g., the following declaration is illegal:

GraphicalObject g (Point (0,0)); // Wrong.
If we were allowed to declare g in this way, then we could attempt to invoke the non-existent member function g.Draw().

In fact, there is a second reason why the declaration of g is an error: The GraphicalObject constructor is not public (see Program gif , lines 15-16). So, even if the GraphicalObject class was a concrete class, we are still prevented from instantiating it. Since the constructor is protected, it can be called by a derived class. Therefore, it is possible to instantiate Circle, Rectangle and Square.


Algorithmic Abstraction

Abstract classes can be used in many interesting ways. One of the most useful paradigms is the use of an abstract class for algorithmic abstraction . The Erase and MoveTo functions defined in Program gif are examples of this.

The Erase and MoveTo functions are implemented in the abstract class GraphicalObject. The algorithms implemented are designed to work in any concrete class derived from GraphicalObject, be it Circle, Rectangle or Square. In effect, we have written algorithms that work regardless of the actual class of the object. Therefore, such algorithms are called abstract algorithms.

Abstract algorithms typically invoke virtual member functions. E.g., the MoveTo function invokes Erase and Draw to do most of the actual work. In this case, the derived classes are expected to inherit the abstract algorithm MoveTo and to override the pure virtual function Draw. Thus, the derived class customizes the behavior of the abstract algorithm by overriding the appropriate member functions. The virtual function resolution mechanism ensures that the ``correct'' member function is always called.


Multiple Inheritance

In C++ a class can be derived from one or more base classes. However, all the base classes must be distinct. I.e., the following declaration is not allowed:

class D : public B, public B { ... }; // Wrong.
Nevertheless, it is possible for a class to be indirectly derived from the same base class more than once. E.g., consider the following class definitions:
class A { ... };
class B1 : public A { ... };
class B2 : public A { ... };
class D : public B1, public B2 { ... };
The derived class D inherits two instances of the base class A--one indirectly via B1, the other indirectly via B2. In particular, this means that D contains two copies of each member variable of class A. It also means that when a member function of A is called, it is necessary to specify which of the two instances of A is to be used--the one from which B1 is derived or the one from which to B2 is derived. This scenario is shown in Figure gif (a).

figure57706
Figure: Multiple Derivation and Virtual Base Classes

Sometimes it makes more sense for the derived class D to contain only one instance of the base class A. In C++ this is accomplished using virtual base classes :

class A { ... };
class B1 : virtual public A { ... };
class B2 : virtual public A { ... };
class D : public B1, public B2 { ... };
In this case, the derived class D contains only one instance of the base class A. Therefore, D contains only one copy of the member variables declared in A and there is not ambiguity when invoking the member functions of A. This corresponds to the situation shown in Figure gif (b).


Run-Time Type Information and Casts

Consider the following declarations which make use of the Rectangle and Square classes defined in Program gif :

Rectangle r (Point (0,0), 5, 10);
Square s (Point (0,0), 15);
Clearly, the assignment
r = s;
is valid because Square is derived from Rectangle. I.e., since a Square is a Rectangle, we may assign s to r.

On the other hand, the assignment

s = r;
is not valid because a Rectangle instance is not necessarily a Square.

Consider now the following declarations:

Rectangle& r = *new Square (Point (0,0), 20);
Square s;
The assignment s=r is still invalid because r is a reference to a Rectangle, and a Rectangle instance is not necessarily a Square, despite the fact that in this case it actually is!

In order to do the assignment, it is necessary to convert the type of r from a reference to a Rectangle to a reference to a Square. This is done in C++ using a cast operator :

s = (Square&) r;
However, a conversion like this is unchecked. Neither the compiler nor the run-time system can determine whether r actually refers to a Square.

To determine the actual type of the object to which r refers, we must make use of run-time type information . In C++ every object instance of a class that has virtual functions keeps track of its type. We can test the type of such an object explicitly using the C++ typeid operator like this:

if (typeid (r) == typeid (Square))
    s = (Square&) r;
This code is type-safe because the cast is only done when the object to which r refers actually is a Square.

C++ also provides a type-safe cast operator called dynamiccast<T>() . In this case, T must either a pointer type or a reference type. The dynamiccast operator combines the run-time type-compatibility check with the type cast operation. For example, the statement

s = dynamic_cast<Square&> (r);
succeeds if r is found at run-time to be is an instance of the Square class. If it is not, the dynamiccast operator throws a badcast exception. (Exceptions are discussed in Section gif ).


Templates

In C++, templates are used to define generic functions and classes. A generic function or class is one which takes a type as an actual parameter. As a result, a single function or class definition gives rise to a family of functions or classes that are compiled from the same source code, but operate on different types.

E.g., consider the following function definition:

int Max (int x, int y)
    { return x > y ? x : y; }
The Max function takes to arguments of type int and returns the larger of the two. To compute the maximum value of a pair doubles, we require a different Max function.

By using a template, we can define a generic Max function like this:

template <class T>
T Max (T x, T y)
    { return x > y ? x : y; }
The template definition gives rise to a family of Max functions, one for every different type T. The C++ compiler automatically creates the appropriate Max functions as needed. E.g., the code sequence
int i = Max (1, 2);
double d = Max (1.0, 2.0);
causes the creation of two Max functions, one for arguments of type int and the other for arguments of type double.

Templates can also be used to define generic classes. E.g., consider the following class definition:

class Stack
{
    // ...
public:
    void Push (int);
    int Pop ();
};
This Stack class represents a stack of integers. The member functions Push and Pop are used to insert and to withdraw an int from the stack (respectively). If instead of ints a stack of doubles is required, we must write a different Stack class. By using a template as shown in Program gif we can define a generic Stack class.

program57992
Program: Stack Template Definition

The required classes are created automatically by the C++ compiler as needed. E.g., the code sequence

Stack<int> s1;
s1.Push (1);
Stack<double> s2;
s2.Push (1.0);
causes the creation of two stack classes, Stack<int> and Stack<double>. This example also illustrates that the name of the class includes the actual type parameter used to create it.


Exceptions

Sometimes unexpected situations arise during the execution of a program. Careful programmers write code that detects errors and deals with them appropriately. However, a simple algorithm can become unintelligible when error-checking is added because the error-checking code can obscure the normal operation of the algorithm.

Exceptions provide a clean way to detect and handle unexpected situations. When a program detects an error, it throws an exception. When an exception is thrown, control is transfered to the appropriate exception handler . By defining a function that catches the exception, the programmer can write the code to handle the error.

In C++, an exception is an object. Typically exceptions are derived (directly or indirectly) from the base class called exception which is defined in the C++ standard library header file called <exception>.

A function throws an exception by using the throw statement:

class domain_error : public exception {};

void f ()
{
    // ...
    throw domain_error ();
}
The throw statement is similar to a return statement. A return statement represents the normal termination of a function and the object returned matches the return value of the function. A throw statement represents the abnormal termination of a function and the object thrown represents the type of error encountered.

Exception handlers are defined using a try block:

void g ()
{
    try {
        f ();
    }
    catch (domain_error) {
        // exception handler
    }
    catch (range_error) {
        // exception handler
    }
}
The body of the try block is executed either until an exception is thrown or until it terminates normally.

One or more exception handlers follow a try block. Each exception handler consists of a catch clause which specifies the exceptions to be caught, and a block of code, which is executed when the exception occurs. When the body of the try block throws an exception for which an exception is defined, control is transfered to the body of the exception handler.

In this example, the exception was thrown by the function f() and caught by the function g(). In general when an exception is thrown, the chain of functions called is searched in reverse (from caller to callee) to find the closest matching catch statement. As the stack of called functions unwinds, the destructors for the local variables declared in those functions are automatically executed. When a program throws an exception that is not caught, the program terminates.


Class Hierarchy Diagrams

figure58095
Figure: Key for the Class Hierarchy Diagrams

figure58102
Figure: Complete Class Hierarchy Diagram


Character Codes

Table: 7-bit ASCII Character Set
 | bits 2-0
bits 6-3 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
0 | NUL | SOH | STX | ETX | EOT | ENQ | ACK | BEL
1 | BS | HT | NL | VT | NP | CR | SO | SI
2 | DLE | DC1 | DC2 | DC3 | DC4 | NAK | SYN | ETB
3 | CAN | EM | SUB | ESC | FS | GS | RS | US
4 | SP | ! | " | # | $ | % | & | '
5 | ( | ) | * | + | , | - | . | /
6 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
7 | 8 | 9 | : | ; | < | = | > | ?
010 | @ | A | B | C | D | E | F | G
011 | H | I | J | K | L | M | N | O
012 | P | Q | R | S | T | U | V | W
013 | X | Y | Z | [ | \ | ] | ^ | _
014 | ` | a | b | c | d | e | f | g
015 | h | i | j | k | l | m | n | o
016 | p | q | r | s | t | u | v | w
017 | x | y | z | { | | | } | ~ | DEL


References

1
Alfred V. Aho, John E. Hopcropft, and Jeffrey D. Ullman. Data Structures and Algorithms. Addison-Wesley, 1983.

2
Alfred V. Aho and Jeffrey D. Ullman. Foundations of Computer Science. Computer Science Press, 1992.

3
ANSI Accredited Standards Committee X3, Information Processing Systems. Working Paper for Draft Proposed International Standard for Information Systems--Programming Language C++, December 1996. Document Number X3J16/96-0225 WG21/N1043.

4
Ken Arnold and James Gosling. The Java Programming Language. The Java Series. Addison-Wesley, 1996.

5
Borland International, 1800 Green Hills Road, P.O. Box 660001, Scotts Valley, CA 95067-0001. Borland C++ Version 3.0 Programmer's Guide, 1991.

6
Timothy A. Budd. Classic Data Structures in C++. Addison-Wesley, 1994.

7
Computational Science Education Project. Mathematical optimization. Virtual book, 1995. http://csep1.phy.ornl.gov/CSEP/MO/MO.html.

8
Computational Science Education Project. Random number generators. Virtual book, 1995. http://csep1.phy.ornl.gov/CSEP/RN/RN.html.

9
Gaelan Dodds de Wolf, Robert J. Gregg, Barbara P. Harris, and Matthew H. Scargill, editors. Gage Canadian Dictionary. Gage Educational Publishing Company, 1997.

10
Rick Decker and Stuart Hirshfeld. Working Classes: Data Structures and Algorithms Using C++. PWS Publishing Company, 1996.

11
Adam Drozdek. Data Structures and Algorithms in C++. PWS Publishing Company, 1996.

12
Margaret A. Ellis and Bjarne Stroustrup. The Annotated C++ Reference Manual. Addison-Wesley, 1990.

13
James A. Field. Makegraph user's guide. Technical Report 94-04, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, 1994.

14
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

15
Michel Goosens, Frank Mittelbach, and Alexander Samarin. The LaTeX Companion. Addison-Wesley, 1994.

16
Irwin Guttman, S. S. WIlks, and J. Stuart Hunter. Introductory Engineering Statistics. John Wiley & Sons, second edition, 1971.

17
Gregory L. Heileman. Data Structures, Algorithms, and Object-Oriented Programming. McGraw-Hill, 1996.

18
Ellis Horowitz and Sartaj Sahni. Data Structures in Pascal. W. H. Freeman and Company, third edition, 1990.

19
Ellis Horowitz, Sartaj Sahni, and Dinesh Mehta. Fundamentals of Data Structures in C++. W. H. Freeman and Company, 1995.

20
Brian W. Kernighan and Dennis M. Ritchie. The C Programming Language. Prentice-Hall, 1978.

21
Leonard Kleinrock. Queueing Systems, Volume I: Theory. John Wiley & Sons, 1975.

22
Donald E. Knuth. Fundamental Algorithms, volume 1 of The Art of Computer Programming. Addison-Wesley, second edition, 1973.

23
Donald E. Knuth. Sorting and Searching, volume 3 of The Art of Computer Programming. Addison-Wesley, 1973.

24
Donald E. Knuth. Seminumerical Algorithms, volume 2 of The Art of Computer Programming. Addison-Wesley, second edition, 1981.

25
Donald E. Knuth. The METAFONTbook. Addison-Wesley, 1986.

26
Donald E. Knuth. The TeXbook. Addison-Wesley, 1986.

27
Elliot B. Koffman, David Stemple, and Caroline E. Wardle. Recommended curriculum for CS2, 1984. Communications of the ACM, 28(8):815-818, August 1985.

28
Leslie Lamport. LaTeX: A Document Preparation System. Addison-Wesley, second edition, 1994.

29
Yedidyah Langsam, Moshe J. Augenstein, and Aaron M. Tenenbaum. Data Structures Using C and C++. Prentice Hall, second edition, 1996.

30
Kenneth McAloon and Anthony Tromba. Calculus, volume 1BCD. Harcourt Brace Jovanovich, Inc., 1972.

31
Thomas L. Naps. Introduction to Program Design and Data Structures. West Publishing, 1993.

32
Stephen K. Park and Keith W. Miller. Random number generators: Good ones are hard to find. Communications of the ACM, 31(10):1192-1201, October 1988.

33
P. J. Plauger. The Draft Standard C++ Library. Prentice Hall, 1995.

34
Stephen R. Schach. Classical and Object-Oriented Software Engineering. Irwin, third edition, 1996.

35
G. Michael Schneider and Steven C. Bruell. Concepts in Data Structures and Software Development. West Publishing, 1991.

36
Bjarne Stroustrup. The C++ Programming Language. Addison-Wesley, second edition, 1991.

37
Bjarne Stroustrup. The Design and Evolution of C++. Addison-Wesley, 1994.

38
Allen B. Tucker, Bruce H. Barnes, Robert M. Aiken, Keith Barker, Kim B. Bruce, J. Thomas Cain, Susan E. Conry, Gerald L. Engel, Richard G. Epstein, Doris K. Lidtke, Michael C. Mulder, Jean B. Rogers, Eugene H. Spafford, and A. Joe Turner. Computing Curricula 1991: Report of the ACM/IEEE-CS Joint Curriculum Task Force. ACM/IEEE, 1991.

39
Larry Wall and Randal L. Schwartz. Programming perl. O'Reilly & Associates, Sebastopol, California, 1991.

40
Mark Allen Weiss. Data Structures and Algorithm Analysis. Benjamin/Cummings, second edition, 1995.

41
Mark Allen Weiss. Algorithms, Data Structures and Problem Solving with C++. Addison-Wesley, 1996.

42
Geoff Whale. Data Structures and Abstraction Using C. Thomson Nelson Australia, 1996.


About Logarithms

In this section we determine the asymptotic behavior of logarithms. Interestingly, despite the fact that [image] diverges as n gets large, [image] for all integers [image]. Hence, [image]. Furthermore, as the following theorem will show, [image] raised to any integer power [image] is still O(n).

Theorem For every integer [image], [image].

extbfProof This result follows immediately from Theorem gif and the observation that for all integers [image],

This observation can be proved by induction as follows:

Base Case Consider the limit

for the case k=1. Using L'Hôpital's rule gif we see that

Inductive Hypothesis Assume that Equation gif holds for [image]. Consider the case k=m+1. Using L'Hôpital's rule we see that

Therefore, by induction on m, Equation gif holds for all integers [image].

For example, using this property of logarithms together with the rule for determining the asymptotic behavior of the product of two functions (Theorem gif ), we can determine that since [image], then [image].


Index

o
seebig oh, seelittle oh
seeEuler's constant
seeomega
seetheta
seelambda
abstract algorithms
Tree Traversals
abstract class
Class Hierarchy, Class Hierarchy, Abstract Classes and Concrete
abstract data type
Foundational Data Structures, Abstract Data Types
abstract solver
Abstract Backtracking Solvers
abstract sorter
Sorting and Sorters
access path
Inserting Items into an
accessor
Array Member Functions, Accessors and Mutators
activation record
The Basic Axioms, The Heap
activity-node graph
Application: Critical Path Analysis
actual parameter
Pass By Value
acyclic
directed graph
Directed Acyclic Graphs
adapter
PreorderInorder and Postorder
address
Abstract Data Types, VariablesPointers and References
address arithmetic
Array Subscript Calculations
adjacency lists
Adjacency Lists
adjacency matrix
Adjacency Matrices
adjacent
Terminology
ADT
seeabstract data type
algorithmic abstraction
Algorithmic Abstraction
ancestor
More Terminology
proper
More Terminology
and
UnionIntersection and Difference
annealing
Simulated Annealing
annealing schedule
Simulated Annealing
arc
directed
Terminology
undirected
Undirected Graphs
arithmetic series
About Arithmetic Series Summation
arithmetic series summation
An Example-Geometric Series Summation, About Arithmetic Series Summation
arity
N-ary Trees
array
Foundational Data Structures
ASCII
Character String Keys
asymptotic behavior
Asymptotic Notation
attributes
Abstract Data Types
AVL balance condition
AVL Search Trees
AVL rotation
Balancing AVL Trees
AVL tree
Basics
B-Tree
B-Trees, B-Trees
Bachmann, P.
An Asymptotic Upper Bound-Big
backtracking algorithms
Backtracking Algorithms
bad_cast
Implementation
bad_cast
Run-Time Type Information and
bag
Projects, Multisets
balance condition
AVL Search Trees, B-Trees
AVL
AVL Search Trees
base class
Class Hierarchy, Derivation and Inheritance
best-fit allocation strategy
Acquiring an Area, Exercises
big oh
An Asymptotic Upper Bound-Big
tightness
Tight Big Oh Bounds, More Notation-Theta and Little
tightness
Tight Big Oh Bounds, More Notation-Theta and Little
transitive property
Properties of Big Oh
binary digit
Binomial Queues
binary heap
Sorting with a Heap
binary operator
Applications
binary search
Locating Items in an , Example-Binary Search
binary search tree
Binary Search Trees, Binary Search Trees
binary tree
Binary Trees, Binary Trees
complete
Complete Trees
binding
Abstract Data Types, VariablesPointers and References
binomial
Binomial Trees
binomial coefficient
Binomial Trees
bit
Binomial Queues
bitsizeof
textbf
bitwise exclusive-or
Buddy System for Storage
Boolean
and
UnionIntersection and Difference
or
UnionIntersection and Difference
bound
Abstract Data Types
branch-and-bound
Branch-and-Bound Solvers
breadth-first spanning tree
Constructing Spanning Trees
breadth-first traversal
Applications, Applications, Breadth-First Traversal, Example-Balancing Scales, Breadth-First Traversal
brute-force algorithms
Brute-Force and Greedy Algorithms
bubble sort
Bubble Sort
bucket sort
Example-Bucket Sort
buckets
Example-Bucket Sort
buddy system
Buddy System for Storage
built-in types
Objects and Classes
C programming language
Abstract Data Types
C++ programming language
Abstract Data Types
carry
Merging Binomial Queues
cast operator
Run-Time Type Information and
ceiling function
About Harmonic Numbers
central limit theorem
Exercises
chained scatter table
Chained Scatter Table
child
Applications, Terminology
circular list
Singly-Linked Lists, Doubly-Linked and Circular Lists
clock frequency
A Simplified Model of
clock period
A Simplified Model of
coalesce
Chained Scatter Table
cocktail shaker sort
Exercises
coefficient
binomial
Binomial Trees
coercion
Canonical Matrix Multiplication
collapsing find
Collapsing Find
column-major order
Exercises
commensurate
elements
Sorted Lists, Basics
elements
Sorted Lists, Basics
functions
More Big Oh Fallacies , More Big Oh Fallacies
functions
More Big Oh Fallacies , More Big Oh Fallacies
complement
Exercises
complete N-ary tree
Complete N-ary Trees
complete binary tree
Complete Trees, Sorting with a Heap
complex numbers
Member Variables and Member
component
connected
Connectedness of an Undirected
compound statement
Rules For Big Oh
concrete class
Class Hierarchy, Abstract Classes and Concrete
conjunction
SetsMultisets and Partitions
connected
undirected graph
Connectedness of an Undirected
connected component
Connectedness of an Undirected , Exercises
conquer
seedivide
const member functions
Array Member Functions
constant
Conventions for Writing Big
constructor
Constructors and Destructors
copy
Copy Constructor
default
Default Constructor
copy constructor
Copy Constructor
counted do loop
Rules For Big Oh
critical activity
Application: Critical Path Analysis
critical path
Application: Critical Path Analysis
critical path analysis
Application: Critical Path Analysis
cubic
Conventions for Writing Big
cycle
More Terminology
negative cost
Single-Source Shortest Path
simple
More Terminology
data ordering property
M-Way Search Trees
database
Associations
decision tree
A Lower Bound on
default constructor
Default Constructor, Default Constructor
default copy constructor
Canonical Matrix Multiplication
degree
Applications
in
Terminology
out
Terminology
dense graph
Sparse vs. Dense Graphs
depth
More Terminology
depth-first spanning tree
Constructing Spanning Trees
depth-first traversal
Example-Balancing Scales, Depth-First Traversal
deque
StacksQueues and Deques, Deques
dereference
Dereferencing Pointers
derivation
Class Hierarchy, Derivation and Inheritance
derivative
Applications
derived class
Derivation and Inheritance
descendant
More Terminology
proper
More Terminology
destructor
Destructors
difference
SetsMultisets and Partitions, Basics, UnionIntersection and Difference
symmetric
Exercises
differentiation
Applications
digit
binary
Binomial Queues
digraph
seedirected graph
Dijkstra's algorithm
Dijkstra's Algorithm
direct containment
Direct vs. Indirect Containment
directed acyclic graph
Directed Acyclic Graphs
directed arc
Terminology
directed graph
Directed Graphs
discrete event simulation
Discrete Event Simulation
disjunction
SetsMultisets and Partitions
distribution sorting
Distribution Sorting
distribution sorts
Sorter Class Hierarchy
divide and conquer
Top-Down Algorithms: Divide-and-Conquer
division method of hashing
Division Method
domain errordomain_error
First and Last Functions
double construction
C++ Magic
double hashing
Double Hashing
double rotation
Double Rotations
double-ended queue
Deques
doubly-linked list
Doubly-Linked and Circular Lists
dual
Application: Critical Path Analysis
dynamic binding
Abstract Data Types
dynamic_cast
Implementation
dynamic_cast
Implementation
dynamic programming
Bottom-Up Algorithms: Dynamic
Programming
dynamic_cast
Run-Time Type Information and
earliest event time
Application: Critical Path Analysis
edge
Applications, Terminology
emanate
Terminology
incident
Terminology
edge-weighted graph
Edge-Weighted and Vertex-Weighted Graphs
element
SetsMultisets and Partitions
emanate
Terminology
enumeration
Projects
equivalence classes
Applications
equivalence of trees
Comparing Trees
equivalence relation
Applications, Kruskal's Algorithm
Euler's constant
About Harmonic Numbers, Solving The Recurrence-Telescoping, Average Running Time
Euler, Leonhard
Binomial Trees
Eulerian walk
Exercises
evaluation stack
Postfix Notation
event-node graph
Application: Critical Path Analysis
exception
Array Subscripting Operator, First and Last Functions
exception handler
Exceptions
exceptions
Exceptions
exchange sorting
Exchange Sorting
exchange sorts
Sorter Class Hierarchy
exclusive or
Character String Keys, Character String Keys
exclusive-or
Buddy System for Storage
exponent
Floating-Point Keys
exponential
Conventions for Writing Big
exponential cooling
Simulated Annealing
exponential distribution
Implementation
expression tree
Expression Trees
external node
N-ary Trees
external path length
Unsuccessful Search
factorial
Analyzing Recursive Functions
feasible solution
Brute-Force Algorithm
Fibonacci hashing method
Fibonacci Hashing
Fibonacci number
Fibonacci Hashing, AVL Search Trees
Fibonacci numbers
Example-Fibonacci Numbers, Example-Computing Fibonacci Numbers
closed-form expression
Example-Fibonacci Numbers
generalized
Example-Generalized Fibonacci Numbers
FIFO
Queues
fifo-in, first-out
Queues
find
collapsing
Collapsing Find
first-fit allocation strategy
Acquiring an Area, Exercises
floor function
About Harmonic Numbers
Floyd's algorithm
Floyd's Algorithm
forest
Binomial Queues, Binomial Queues, Implementing a Partition using
formal parameter
Pass By Value
Fortran
Abstract Data Types
foundational data structure
Foundational Data Structures
free
Singly Linked Free Storage
free list
Singly Linked Free Storage
free store
Array Constructor
frexp
Floating-Point Keys
friend
Canonical Matrix Multiplication, Iterator
fully connected graph
Exercises
Gauss, Karl Friedrich
Binomial Trees
generalization
Deques
generalized Fibonacci numbers
Example-Generalized Fibonacci Numbers
generic
Templates
geometric series
About Geometric Series Summation
geometric series summation
An Example-Geometric Series Summation, Example-Geometric Series Summation Again, About Geometric Series Summation, Example-Geometric Series Summation Yet
golden ratio
Fibonacci Hashing
graph
connectedness
Connectedness of an Undirected
dense
Sparse vs. Dense Graphs
directed
Directed Graphs
directed acyclic
Directed Acyclic Graphs
edge-weighted
Edge-Weighted and Vertex-Weighted Graphs
labeled
Labeled Graphs
sparse
Sparse vs. Dense Graphs
traversal
Graph Traversals
undirected
Undirected Graphs
vertex-weighted
Edge-Weighted and Vertex-Weighted Graphs
graph theory
Graphs and Graph Algorithms
harmonic number
Average Running Times, About Harmonic Numbers, Average Case Analysis, Solving The Recurrence-Telescoping, Average Running Time
harmonic series
About Harmonic Numbers
hash function
Keys and Hash Functions, Keys and Hash Functions
hash table
Hash Tables
hashing
division method
Division Method
Fibonacci method
Fibonacci Hashing
middle-square method
Middle Square Method
multiplication method
Multiplication Method
head
Singly-Linked Lists
heap
Array Constructor, Basics, The Heap
heapify
Sorting with a Heap
heapsort
Sorting with a Heap
height
of a node in a tree
More Terminology
of a tree
More Terminology
heuristic
Depth-FirstBranch-and-Bound Solver
hierarchy
Trees
Horner's rule
Another Example-Horner's Rule, Example-Geometric Series Summation Again, Character String Keys
idempotent
Objects
in-degree
Terminology, Topological Sort
in-place sorting
Insertion Sorting, Selection Sorting
incident
Terminology
increment
Generating Random Numbers
indirect containment
Direct vs. Indirect Containment, General Trees
infix
Applications
infix notation
Infix Notation
inheritance
Derivation and Inheritance
initialized static variables
The Heap
inorder traversal
Inorder Traversal, Traversing a Search Tree
M-way tree
Traversing a Search Tree
insertion sorting
Insertion Sorting
straight
Straight Insertion Sort
insertion sorts
Sorter Class Hierarchy
integral type
Integral Keys
interface
Class Hierarchy
internal node
N-ary Trees
internal path length
Unsuccessful Search
complete binary tree
Complete Trees
internal path length of a tree
Successful Search
Internet domain name
Character String Keys
intersection
SetsMultisets and Partitions, Basics, UnionIntersection and Difference
inverse modulo W
Multiplication Method
inversion
Average Running Time
isomorphic
Alternate Representations for Trees
isomorphic trees
Exercises
iterative algorithm
Example-Fibonacci Numbers
Java
Projects
key
Associations, Keys and Hash Functions
keyed data
Using Associations
knapsack problem
Example-0/1 Knapsack Problem
Kruskal's algorithm
Kruskal's Algorithm
L'Hôpital's rule
About Logarithms, About Logarithms
l-value
Abstract Data Types, VariablesPointers and References
labeled graph
Labeled Graphs
lambda
seeload factor
last-in, first-out
Stacks
latest event time
Application: Critical Path Analysis
leaf
Terminology
leaf node
N-ary Trees
least-significant-digit-first radix sorting
Radix Sort
left subtree
Binary Trees, M-Way Search Trees
leftist tree
Leftist Trees
level
More Terminology
level-order
Complete N-ary Trees
level-order traversal
Applications
lexicographic order
Array Subscript Calculations
lexicographic ordering
Radix Sort
lexicographically precede
Radix Sort
lifetime
Abstract Data Types, Abstract Data Types, VariablesPointers and References
LIFO
Stacks
limit
Properties of Big Oh
linear
Conventions for Writing Big
linear congruential random number generator
Generating Random Numbers
linear probing
Linear Probing
linear search
Yet Another Example-Finding the
linked list
Foundational Data Structures
list
Ordered Lists and Sorted
little oh
More Notation-Theta and Little
LL rotation
Single Rotations
in a B-tree
Removing Items from a
load factor
Average Case Analysis
log squared
Conventions for Writing Big
logarithm
Conventions for Writing Big
loop
More Terminology
loose asymptotic bound
More Notation-Theta and Little
LR rotation
Double Rotations
ukasiewicz, Jan
Applications
M-way search tree
M-Way Search Trees
mantissa
Floating-Point Keys
many-to-one mapping
Keys and Hash Functions
matrix
adjacency
Adjacency Matrices
sparse
Adjacency Matrices
max-heap
Sorting with a Heap
median
Selecting the Pivot
median-of-three pivot selection
Selecting the Pivot
member variable accessor
Array Member Functions
memory map
Basics
merge sort
Example-Merge Sorting
merge sorting
Merge Sorting
merge sorts
Sorter Class Hierarchy
mergeable priority queue
Basics
merging nodes in a B-tree
Removing Items from a
Mersenne primes
The Minimal Standard Random
middle-square hashing method
Middle Square Method
min heap
Basics
minimal subgraph
Minimum-Cost Spanning Trees
minimum spanning tree
Minimum-Cost Spanning Trees
mixed linear congruential random number generator
Generating Random Numbers
modulus
Generating Random Numbers
Monte Carlo methods
Monte Carlo Methods
multi-dimensional array
Multi-Dimensional Arrays
multiple inheritance
Containers, Derivation and Inheritance
multiplication hashing method
Multiplication Method
multiplicative linear congruential random number generator
Generating Random Numbers
multiset
Multisets
mutator
Mutators
N-ary tree
N-ary tree
N-ary Trees
N-queens problem
N-queens problem
Exercises
name
Abstract Data Types, Abstract Data Types, Abstract Data Types, VariablesPointers and References
name()
Container Class Default Put
Nary tree
textbf
negative cost cycle
Single-Source Shortest Path
Newton, Isaac.
Binomial Trees
next-fit allocation strategy
Exercises
node
Applications, Basics, N-ary Trees, Binary Trees, Terminology
non-recursive algorithm
Example-Fibonacci Numbers
normalize
Generating Random Numbers
null path length
Leftist Trees, Leftist Trees
null-terminated sequence
Character String Keys
object
Objects and Classes
object-oriented programming
Abstract Data Types
object-oriented programming language
Abstract Data Types
objective function
Brute-Force Algorithm
odd-even transposition sort
Exercises
omega
An Asymptotic Lower Bound-Omega
open addressing
Scatter Table using Open
operator overloading
Prefix Notation, Mutators
operator precedence
Applications
optimal binary search tree
Exercises
or
UnionIntersection and Difference
ordered list
Ordered Lists and Sorted , Ordered Lists
ordered tree
N-ary Trees, Binary Trees
ordinal number
Positions of Items in
oriented tree
N-ary Trees
out_of_range
Array Subscripting Operator
out-degree
Terminology
overloading
operator
Mutators
override
Derivation and Inheritance, Derivation and Inheritance
ownership
Ownership of Contained Objects
parameter passing
Parameter Passing
parent
Applications, Terminology
parentheses
Applications
partial order
Comparing Sets
partition
Partitions, Kruskal's Algorithm
Pascal
Abstract Data Types
Pascal's triangle
Example-Computing Binomial Coefficients
Pascal, Blaise
Example-Computing Binomial Coefficients
pass-by-reference
Parameter Passing, Pass By Reference
pass-by-value
Parameter Passing
path
Terminology
access
Inserting Items into an
path length
external
Unsuccessful Search
internal
Unsuccessful Search
weighted
Shortest-Path Algorithms
perfect binary tree
Searching a Binary Tree, AVL Search Trees
period
Generating Random Numbers
pivot
Quicksort
placement syntax
Working with Multiple Storage
pointer
Pointers Are Variables
Polish notation
Applications
polymorphism
Class Hierarchy, Polymorphism
polynomial
About Polynomials, About Polynomials Again
position
Ordered Lists
postorder traversal
Postorder Traversal
power set
Array and Bit-Vector Sets
precede lexicographically
Radix Sort
predecessor
Member Variables, More Terminology
prefix notation
Prefix Notation
preorder traversal
Preorder Traversal
prepend
Prepend
Prim's algorithm
Prim's Algorithm
primary clustering
Linear Probing
prime
relatively
Multiplication Method
priority queue
mergeable
Basics
probability density function
Implementation
probe sequence
Scatter Table using Open
processor stack
The Heap
proper subset
Comparing Sets
proper superset
Comparing Sets
protected
Canonical Matrix Multiplication
pruning a solution space
Branch-and-Bound Solvers
pseudorandom
Generating Random Numbers
pure virtual function
Abstract Classes and Concrete
pure virtual member function
Class Hierarchy
quadratic
Conventions for Writing Big
quadratic probing
Quadratic Probing
queue
StacksQueues and Deques
quicksort
Quicksort
r-value
Abstract Data Types, VariablesPointers and References
radix sorting
Radix Sort
random number generator
linear congruential
Generating Random Numbers
mixed linear congruential
Generating Random Numbers
multiplicative linear congruential
Generating Random Numbers
random numbers
Generating Random Numbers
random variable
Random Variables
rank
Union by Height or
read-only access
Array Member Functions
record
Abstract Data Types
recurrence relation
Analyzing Recursive Functions
recursive algorithm
Analyzing Recursive Functions, Example-Fibonacci Numbers
reference
References are Not Variables
reflexive
Applications
relation
equivalence
Applications
relatively prime
Multiplication Method
repeated substitution
Solving Recurrence Relations-Repeated Substitution
reserved
Singly Linked Free Storage
Reverse-Polish notation
Applications
right subtree
Binary Trees
RL rotation
Double Rotations
root
Basics
rotation
AVL
Balancing AVL Trees
double
Double Rotations
LL
Single Rotations, Removing Items from a
LL
Single Rotations, Removing Items from a
LR
Double Rotations
RL
Double Rotations
RR
Single Rotations, Removing Items from a
RR
Single Rotations, Removing Items from a
single
Double Rotations
row-major order
Array Subscript Calculations
RPN
seeReverse-Polish notation
RR rotation
Single Rotations
in a B-tree
Removing Items from a
RTTI
seerun-time type information
run-time type information
Implementation, Container Class Default Put , Run-Time Type Information and
scales
Example-Balancing Scales
scatter tables
Scatter Tables
scope
Abstract Data Types, Abstract Data Types, VariablesPointers and References
search tree
M-way
M-Way Search Trees
binary
Binary Search Trees
seed
Generating Random Numbers
selection sorting
Selection Sorting
selection sorts
Sorter Class Hierarchy
sentinel
Singly-Linked Lists, Adjacency Matrices
separate chaining
Separate Chaining
sequence
Ordered Lists
set
SetsMultisets and Partitions
sibling
Terminology
significant
Floating-Point Keys
simple cycle
More Terminology
simulated annealing
Simulated Annealing
simulation time
Discrete Event Simulation
single rotation
Double Rotations
single-ended queue
Queues
singleton
The NullObject Singleton Class, Exercises, Implementation
singly-linked list
Doubly-Linked and Circular Lists
size
Abstract Data Types, VariablesPointers and References
slack time
Application: Critical Path Analysis
solution space
Example-Balancing Scales
solver
Abstract Backtracking Solvers
sort
topological
Topological Sort
sorted list
Ordered Lists and Sorted , Sorted Lists, Basics
sorter
Sorting and Sorters
sorting
in place
Selection Sorting
in-place
Insertion Sorting
sorting algorithm
bucket sort
Example-Bucket Sort
sorting by distribution
Distribution Sorting
sorting by exchanging
Exchange Sorting
sorting by insertion
Insertion Sorting
sorting by merging
Merge Sorting
sorting by selection
Selection Sorting
source
Exercises
spanning tree
Minimum-Cost Spanning Trees
breadth-first
Constructing Spanning Trees
depth-first
Constructing Spanning Trees
minimum
Minimum-Cost Spanning Trees
sparse graph
Sparse vs. Dense Graphs
sparse matrix
Adjacency Matrices
specialization
Deques
stable sorts
Basics
stack
Stacks
stack frame
The Basic Axioms
state
Discrete Event Simulation
static binding
Abstract Data Types
static member
Implementation
Stirling numbers
Partitions, Partitions
straight insertion sorting
Straight Insertion Sort
straight selection sorting
Straight Selection Sorting
string
Character String Keys
strongly connected
Connectedness of a Directed
subgraph
Minimum-Cost Spanning Trees
minimal
Minimum-Cost Spanning Trees
subset
Comparing Sets
proper
Comparing Sets
subtraction
SetsMultisets and Partitions
subtree
Applications
successor
Member Variables, More Terminology
superset
Comparing Sets
proper
Comparing Sets
symbol table
HashingHash Tables and , Applications
symmetric
Applications
symmetric difference
Exercises
tail
Singly-Linked Lists, Singly-Linked Lists
telescoping
Solving The Recurrence-Telescoping, Running Time of Divide-and-Conquer
temperature
Simulated Annealing
template
Templates
tertiary tree
N-ary Trees
theta
More Notation-Theta and Little
throw
Exceptions
tight asymptotic bound
Tight Big Oh Bounds
time
simulation
Discrete Event Simulation
topological sort
Topological Sort
total order
Basics
binary trees
Comparing Trees
transitive
Sorted Lists, Applications, Basics
traversal
Tree Traversals, Example-Balancing Scales, Graph Traversals
breadth-first
Breadth-First Traversal, Breadth-First Traversal
breadth-first
Breadth-First Traversal, Breadth-First Traversal
depth-first
Depth-First Traversal
inorder
Inorder Traversal, Traversing a Search Tree
inorder
Inorder Traversal, Traversing a Search Tree
postorder
Postorder Traversal
preorder
Preorder Traversal
tree
Basics
N-ary
N-ary Trees
binary
Binary Trees
equivalence
Comparing Trees
expression
Expression Trees
height
More Terminology
internal path length
Successful Search
leftist
Leftist Trees
ordered
N-ary Trees, Binary Trees
ordered
N-ary Trees, Binary Trees
oriented
N-ary Trees
search
seesearch tree
tertiary
N-ary Trees
traversal
Tree Traversals
tree traversal
Applications
type
Abstract Data Types, VariablesPointers and References
typeid
Container Class Default Put
typeinfo.h
Container Class Default Put
undirected arc
Undirected Graphs
undirected graph
Undirected Graphs
uniform distribution
Spreading Keys Evenly
uniform hashing model
Average Case Analysis
union
PushPop, and Top , SetsMultisets and Partitions, Basics, UnionIntersection and Difference
union by rank
Union by Height or
union by size
Union by Size
universal set
SetsMultisets and Partitions, Kruskal's Algorithm
unsorted list
Basics
value
Abstract Data Types, Associations, VariablesPointers and References
variable
VariablesPointers and References
Venn diagram
Alternate Representations for Trees, SetsMultisets and Partitions
vertex
Terminology
vertex-weighted graph
Edge-Weighted and Vertex-Weighted Graphs
virtual
Class Hierarchy
virtual base class
Multiple Inheritance
visibility
Abstract Data Types
weakly connected
Connectedness of a Directed
weighted path length
Shortest-Path Algorithms
word size
Middle Square Method
worst-fig allocation strategy
Exercises
wrapper
Object Wrappers for the

next up previous contents


Tight Big Oh Bounds

Big oh notation characterizes the asymptotic behavior of a function by providing an upper bound on the rate at which the function grows as n gets large. Unfortunately, the notation does not tell us how close the actual behavior of the function is to the bound. I.e., the bound might be very close (tight) or it might be overly conservative (loose).

The following definition tells us what makes a bound tight, and how we can test to see whether a given asymptotic bound is the best one available.

Definition (Tightness) Consider a function f(n)=O(g(n)). If for every function h(n) such that f(n)=O(h(n)) it is also true that g(n)=O(h(n)), then we say that g(n) is a tight asymptotic bound on f(n).

For example, consider the function f(n)=8n+128. In Section gif , it was shown that [image]. However, since f(n) is a polynomial in n, Theorem gif tells us that f(n)=O(n). Clearly O(n) is a tighter bound on the asymptotic behavior of f(n) than is [image].

By Definition gif , in order to show that g(n)=n is a tight bound on f(n), we need to show that for every function h(n) such that f(n)=O(h(n)), it is also true that g(n)=O(h(n)).

We will show this result using proof by contradiction: Assume that g(n) is not a tight bound for f(n)=8n+128. Then there exists a function h(n) such that f(n)=8n+128=O(h(n)), but for which [image]. Since 8n+128=O(h(n)), by the definition of big oh there exist positive constants c and [image] such that [image].

Clearly, for all [image], [image]. Therefore, [image]. But then, by the definition of big oh, we have the g(n)=O(h(n))--a contradiction! Therefore, the bound f(n)=O(n) is a tight bound.


More Big Oh Fallacies and Pitfalls

The purpose of this section is to dispel some common misconceptions about big oh. The next fallacy is related to the selection of the constants c and [image] used to show a big oh relation.

Fallacy Consider non-negative functions f(n), [image], and [image], such that [image]. Since [image] for all integers [image] if [image], then by Definition gif [image].

This fallacy often results from the following line of reasoning: Consider the function [image]. Let [image] and [image]. Then f(n) must be O(n), since [image]. However, this line of reasoning is false because according to Definition gif , c must be a positive constant, not a function of n.

The next fallacy involves a misunderstanding of the notion of the asymptotic upper bound.

Fallacy Given non-negative functions [image], [image], [image], and [image], such that [image], [image], and for all integers [image], [image], then [image].

This fallacy arises from the following line of reasoning. Consider the function [image] and [image]. Since [image] for all values of [image], we might be tempted to conclude that [image]. In fact, such a conclusion is erroneous. E.g., consider [image] and [image]. Clearly, the former is [image] and the latter is [image]. Clearly too, [image] for all values of [image]!

The previous fallacy essentially demonstrates that while we may know how the asymptotic upper bounds on two functions are related, we don't necessarily know, in general, the relative behavior of the two bounded functions.

This fallacy often arises in the comparison of the performance of algorithms. Suppose we are comparing two algorithms, A and B, to solve a given problem and we have determined that the running times of these algorithms are [image] and [image], respectively. Fallacy gif demonstrates that it is an error to conclude from the fact that [image] that algorithm A will solve the problem faster than algorithm B for all problem sizes.

But what about any one specific problem size? Can we conclude that for a given problem size, say [image], that algorithm A is faster than algorithm B? The next fallacy addresses this issue.

Fallacy Given non-negative functions [image], [image], [image], and [image], such that [image], [image], and for all integers [image], [image], there exists an integer [image] for which then [image].

This fallacy arises from a similar line of reasoning as the preceding one. Consider the function [image] and [image]. Since [image] for all values of [image], we might be tempted to conclude that there exists a value [image] for which [image]. Such a conclusion is erroneous. E.g., consider [image] and [image]. Clearly, the former is [image] and the latter is [image]. Clearly too, since [image] for all values of [image], there does not exist any value [image] for which [image].

The final fallacy shows that not all functions are commensurate :

Fallacy Given two non-negative functions f(n) and g(n) then either f(n)=O(g(n)) or g(n)=O(f(n)).

This fallacy arises from thinking that the relation [image] is like [image] and can be used to compare any two functions. However, not all functions are commensurate. gif Consider the following functions:

Clearly, there does not exist a constant c for which [image] for any even integer n, since the g(n) is zero and f(n) is not. Conversely, there does not exist a constant c for which [image] for any odd integer n, since the f(n) is zero and g(n) is not. Hence, neither f(n)=O(g(n)) nor g(n)=O(f(n)) is true.


Conventions for Writing Big Oh Expressions

Certain conventions have evolved which concern how big oh expressions are normally written:

  • First, it is common practice when writing big oh expressions to drop all but the most significant terms. Thus, instead of [image] we simply write [image].
  • Second, it is common practice to drop constant coefficients. Thus, instead of [image], we simply write [image]. As a special case of this rule, if the function is a constant, instead of, say O(1024), we simply write O(1).

Of course, in order for a particular big oh expression to be the most useful, we prefer to find a tight asymptotic bound (see Definition gif ). For example, while it is not wrong to write [image], we prefer to write f(n)=O(n), which is a tight bound.

Certain big oh expressions occur so frequently that they are given names. Table gif lists some of the commonly occurring big oh expressions and the usual name given to each of them.

Table: The Names of Common Big Oh Expressions
expression | name
O(1) | constant
 | logarithmic
 | log squared
O(n) | linear
 | n log n
 | quadratic
 | cubic
 | exponential


An Asymptotic Lower Bound-Omega

The big oh notation introduced in the preceding section is an asymptotic upper bound. In this section, we introduce a similar notation for characterizing the asymptotic behavior of a function, but in this case it is a lower bound.

Definition (Omega) Consider a function f(n) which is non-negative for all integers [image]. We say that ``f(n) is omega g(n),'' which we write [image], if there exists an integer [image] and a constant c>0 such that for all integers [image], [image].

The definition of omega is almost identical to that of big oh. The only difference is in the comparison--for big oh it is [image]; for omega, it is [image]. All of the same conventions and caveats apply to omega as they do to big oh.

  • A Simple Example
  • About Polynomials Again


A Simple Example

Consider the function [image] which is shown in Figure gif . Clearly, f(n) is non-negative for all integers [image]. We wish to show that [image]. According to Definition gif , in order to show this we need to find an integer [image] and a constant c>0 such that for all integers [image], [image].

As with big oh, it does not matter what the particular constants are--as long as they exist! E.g., suppose we choose c=1. Then

Since [image] for all values of [image], we conclude that [image].

So, we have that for c=1 and [image], [image] for all integers [image]. Hence, [image]. Figure gif clearly shows that the function [image] is less than the function f(n)=5n-64n+256 for all values of [image]. Of course, there are many other values of c and [image] that will do. For example, c=2 and [image].

figure1665
Figure: Showing that [image].


About Polynomials Again

In this section we reexamine the asymptotic behavior of polynomials in n. In Section gif we showed that [image]. That is, f(n) grows asymptotically no more quickly than [image]. This time we are interested in the asymptotic lower bound rather than the asymptotic upper bound. We will see that as n gets large, the term involving [image] also dominates the lower bound in the sense that f(n) grows asymptotically as quickly as [image]. I.e., that [image].

Theorem Consider a polynomial in n of the form

where [image]. Then [image].

extbfProof We begin by taking the term [image] out of the summation:

Since, n is a non-negative integer and [image], the term [image] is positive. For each of the remaining terms in the summation, [image]. Hence

Note that for integers [image], [image] for [image]. Thus

Consider the term in parentheses on the right. What we need to do is to find a positive constant c and an integer [image] so that for all integers [image] this term is greater than or equal to c:

We choose the value [image] for which the term is greater than zero:

The value [image] will suffice! Thus

From Equation gif we see that we have found the constants [image] and c, such that for all [image], [image]. Thus, [image].

This property of the asymptotic behavior of polynomials is used extensively. In fact, whenever we have a function, which is a polynomial in n, [image] we will immediately ``drop'' the less significant terms (i.e., terms involving powers of n which are less than m), as well as the leading coefficient, [image], to write [image].


More Notation-Theta and Little Oh

This section presents two less commonly used forms of asymptotic notation. They are:

  • A notation, [image], to describe a function which is both O(g(n)) and [image], for the same g(n). (Definition gif ).
  • A notation, [image], to describe a function which is O(g(n)) but not [image], for the same g(n). (Definition gif ).

Definition (Theta) Consider a function f(n) which is non-negative for all integers [image]. We say that ``f(n) is theta g(n),'' which we write [image], if and only if f(n) is O(g(n)) and f(n) is [image].

Recall that we showed in Section gif that a polynomial in n, say [image], is [image]. We also showed in Section gif that a such a polynomial is [image]. Therefore, according to Definition gif , we will write [image].

Definition (Little Oh) Consider a function f(n) which is non-negative for all integers [image]. We say that ``f(n) is little oh g(n),'' which we write f(n)=o(g(n)), if and only if f(n) is O(g(n)) but f(n) is not [image].

Little oh notation represents a kind of loose asymptotic bound in the sense that if we are given that f(n)=o(g(n)), then we know that g(n) is an asymptotic upper bound since f(n)=O(g(n)), but g(n) is not an asymptotic lower bound since f(n)=O(g(n)) and [image] implies that [image]. gif

For example, consider the function f(n)=n+1. Clearly, [image]. Clearly too, [image], since not matter what c we choose, for large enough n, [image]. Thus, we may write [image].


Suggested Course Outline

This text may be used in either a one semester or a two semester course. The course which I teach at Waterloo is a one-semester course that comprises 36 lecture hours on the following topics:

  1. Review of the fundamentals of programming in C++ and an overview of object-oriented programming with C++. (Appendix gif ). [4 lecture hours].
  2. Models of the computer, algorithm analysis and asymptotic notation (Chapters gif and gif ). [4 lecture hours].
  3. Foundational data structures, abstraction and abstract data types (Chapters gif and gif ). [4 lecture hours].
  4. Stacks, queues, ordered lists and sorted lists (Chapters gif and gif ). [3 lecture hours].
  5. Hashing, hash tables and scatter tables (Chapter gif ). [3 lecture hours].
  6. Trees and search trees (Chapters gif and gif ). [6 lecture hours].
  7. Heaps and priority queues (Chapter gif ). [3 lecture hours].
  8. Algorithm design techniques (Chapter gif ). [3 lecture hours].
  9. Sorting algorithms and sorters (Chapter gif ). [3 lecture hours].
  10. Graphs and graph algorithms (Chapter gif ). [3 lecture hours].

Depending on the background of students, a course instructor may find it necessary to review features of the C++ language. For example, an understanding of templates is required for the foundational data structures discussed in Chapter gif . Similarly, students need to understand the workings of classes and inheritance in order to understand the unifying class hierarchy discussed in Chapter gif .


Asymptotic Analysis of Algorithms

The previous chapter presents a detailed model of the computer which involves a number of different timing parameters-- [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], and [image]. We show that keeping track of the details is messy and tiresome. So we simplify the model by measuring time in clock cycles, and by assuming that each of the parameters is equal to one cycle. Nevertheless, keeping track of and carefully counting all the cycles is still a tedious task.

In this chapter we introduce the notion of asymptotic bounds, principally big oh, and examine the properties of such bounds. As it turns out, the rules for computing and manipulating big oh expressions greatly simplify the analysis of the running time of a program when all we are interested in is its asymptotic behavior.

For example, consider the analysis of the running time of Program gif , which is just Program gif again, an algorithm to evaluate a polynomial using Horner's rule.

program1841
Program: Program gif Again

Table: Computing the running time of Program gif
statement | detailed model | simple | big oh
 |  | model |
3 |  | 5 | O(1)
4a |  | 4 | O(1)
4b |  | 3n+3 | O(n)
4c |  | 4n | O(n)
5 |  | 9n | O(n)
6 |  | 2 | O(1)
TOTAL |  | 16n+14 | O(n)

Table gif shows the running time analysis of Program gif done in three ways--a detailed analysis, a simplified analysis, and an asymptotic analysis. In particular, note that all three methods of analysis are in agreement: Statements 3, 4a, and 6 execute in a constant amount of time; 4b, 4c, and 5 execute in an amount of time which is proportional to n, plus a constant.

The most important observation to make is that, regardless of what the actual constants are, the asymptotic analysis always produces the same answer! Since the result does not depend upon the values of the constants, the asymptotic bound tells us something fundamental about the running time of the algorithm. And this fundamental result does not depend upon the characteristics of the computer and compiler actually used to execute the program!

Of course, you don't get something for nothing. While the asymptotic analysis may be significantly easier to do, all that we get is an upper bound on the running time of the algorithm. In particular, we know nothing about the actual running time of a particular program. (Recall Fallacies gif and gif ).

  • Rules For Big Oh Analysis of Running Time
  • Example-Prefix Sums
  • Example-Fibonacci Numbers
  • Example-Bucket Sort
  • Reality Check
  • Checking Your Analysis


Rules For Big Oh Analysis of Running Time

In this section we present some simple rules for determining a big-oh upper bound on the running time of the basic compound statements in a C++ program.

Rule gif follows directly from Theorem gif . The total running time of a sequence of statements is equal to the sum of the running times of the individual statements. By Theorem gif , when computing the sum of a series of functions it is the largest one (the [image]) that determines the bound.

Rule gif appears somewhat complicated due to the semantics of the C++ for statement. However, it follows directly from Theorem gif . Consider the following simple counted do loop .

for (int i = 0; i < n; ++i)
       ;
Here [image] is int i = 0, so its running time is constant ( [image]); [image] is i < n, so its running time is constant ( [image]); and [image] is ++i, so its running time is constant ( [image]). Also, the number of iterations is I(n)=n. According to Rule gif , the running time of this is [image], which simplifies to [image]. Furthermore, if the loop body does anything at all, its running time must be [image]. Hence, the loop body will dominate the calculation of the maximum, and the running time of the loop is simply [image].

If we don't know the exact number of iterations executed, I(n), we can still use Rule gif provided we have an upper bound, I(n)=O(f(n)), on the number of iterations executed. In this case, the running time is [image].

Rule gif follows directly from the observation that the total running time for an if-then-else statement will never exceed the sum of the running time of the conditional test, [image], plus the larger of the running times of the then part, [image], and the else part, [image].


Example-Prefix Sums

In this section, we will determine a tight big-oh bound on the running time of a program to compute the series of sums [image], [image], ..., [image], where

An algorithm to compute this series of summations is given in Program gif . Table gif summarizes the running time calculation.

program1924
Program: Program to compute [image] for

Table: Computing the running time of Program gif
statement | time
3a | O(1)
3b |
3c |
5 |
6a |
6b |
6c |
7 |
8 |
TOTAL |

Usually the easiest way to analyze program which contains nested loops is to start with the body of the inner-most loop. In Program gif , the inner-most loop comprises lines 6 and 7. In all, a constant amount of work is done--this includes the loop body (line 7), the conditional test (line 6b) and the incrementing of the loop index (line 6c).

For a given value of j, the inner-most loop is done a total j+1 times. And since the outer loop is done for [image], in the worst case, the inner-most loop is done n times. Therefore, the contribution of the inner loop to the running time of one iteration of the outer loop is O(n).

The rest of the outer loop (lines 3, 5 and 8) does a constant amount of work in each iteration. This constant work is dominated by the O(n) of the inner loop. The outer loop is does exactly n iterations. Therefore, the total running time of the program is [image].

But is this a tight big oh bound? We might suspect that it is not, because of the worst-case assumption we made in the analysis concerning the number of times the inner loop is executed. The inner-most loop is done exactly j+1 times for [image]. However, we did the calculation assuming the inner loop is done O(n) times, in each iteration of the outer loop. Unfortunately, in order to determine whether our answer is a tight bound, we must determine more precisely the actual running time of the program.

However, there is one approximate calculation that we can easily make. If we observe that the running time will be dominated by the work done in the inner-most loop, and that the work done in one iteration of the inner-most loop is constant, then all we need to do is to determine exactly the number of times the inner loop is actually executed. This is given by:

Therefore, the result [image] is a tight, big-oh bound on the running time of Program gif .


Example-Fibonacci Numbers

In this section we will compare the asymptotic running times of two different programs that both compute Fibonacci numbers. gif The Fibonacci numbers are the series of numbers [image], [image], ..., given by

Fibonacci numbers are interesting because they seem to crop up in the most unexpected situations. However, in this section, we are merely concerned with writing an algorithm to compute [image] given n.

Fibonacci numbers are easy enough to compute. Consider the sequence of Fibonacci numbers

The next number in the sequence is computed simply by adding together the last two numbers--in this case it would be 55=21+34. Program gif is a direct implementation of this idea. The running time of this algorithm is clearly O(n) as shown by the analysis in Table gif .

program1972
Program: Non-recursive program to compute Fibonacci numbers

Table: Computing the running time of Program gif
statement | time
3 | O(1)
4 | O(1)
5a | O(1)
5b |
5c |
7 |
8 |
9 |
11 | O(1)
TOTAL | O(n)

Recall that the Fibonacci numbers are defined recursively: . However, the algorithm used in Program gif is non-recursive --it is iterative . What happens if instead of using the iterative algorithm, we use the definition of Fibonacci numbers to implement directly a recursive algorithm ? Such an algorithm is given in Program gif and its running time is summarized in Table gif .

program2002
Program: Recursive program to compute Fibonacci numbers

Table: Computing the running time of Program gif
 | time
statement | n<2 |
3 | O(1) | O(1)
4 | O(1) | --
6 | -- | T(n-1)+T(n-2)+O(1)
TOTAL | O(1) | T(n-1)+T(n-2)+O(1)

From Table gif we find that the running time of the recursive Fibonacci algorithm is given by the recurrence

But how do you solve a recurrence containing big oh expressions?

It turns out that there is a simple trick we can use to solve a recurrence containing big oh expressions as long as we are only interested in an asymptotic bound on the result. Simply drop the [image] s from the recurrence, solve the recurrence, and put the [image] back! In this case, we need to solve the recurrence

In the previous chapter, we used successfully repeated substitution to solve recurrences. However, in the previous chapter, all of the recurrences only had one instance of [image] on the right-hand-side--in this case there are two. There is something interesting about this recurrence: It looks very much like the definition of the Fibonacci numbers. In fact, we can show by induction on n that [image].

extbfProof (By induction).

Base Case There are two base cases:

Inductive Hypothesis Suppose that [image] for [image] for some [image]. Then

Hence, by induction on k, [image] for all [image].

So, we can now say with certainty that the running time of the recursive Fibonacci algorithm, Program gif , is [image]. But is this good or bad? The following theorem shows us how bad this really is!

Theorem (Fibonacci numbers) The Fibonacci numbers are given by the closed form expression

where [image] and [image].

extbfProof (By induction).

Base Case There are two base cases:

Inductive Hypothesis Suppose that Equation gif holds for [image] for some [image]. First, we make the following observation:

Similarly,

Now, we can show the main result:

Hence, by induction, Equation gif correctly gives [image] for all [image].

Theorem gif gives us that [image] where [image] and [image]. Consider [image]. A couple of seconds with a calculator should suffice to convince you that [image]. Consequently, as n gets large, [image] is vanishingly small. Therefore, [image]. In asymptotic terms, we write [image]. Now, since [image], we can write that [image].

Returning to Program gif , recall that we have already shown that its running time is [image]. And since [image], we can write that [image]. I.e., the running time of the recursive Fibonacci program grows exponentially with increasing n. And that is really bad in comparison with the linear running time of Program gif !

Figure gif shows the actual running times of both the non-recursive and recursive algorithms for computing Fibonacci numbers. gif Because 32-bit unsigned integers are used, it is only possible to compute Fibonacci numbers up to [image] before overflowing.

The graph shows that up to about n=35, the running times of the two algorithms are comparable. However, as n increases past 40, the exponential growth rate of Program gif is clearly evident. In fact, the actual time taken by Program gif to compute [image] was in excess of one hour!

figure2138
Figure: Actual Running Times of Programs gif and gif


Example-Bucket Sort

So far all of the asymptotic running time analyses presented in this chapter have resulted in tight big oh bounds. In this section we consider an example which illustrates that a cursory big oh analysis does not always result in a tight bound on the running time of the algorithm.

In this section we consider an algorithm to solve the following problem: Sort an array of n integers [image], [image], ..., [image], each of which is known to be between 0 and m-1 for some fixed m. I.e., [image]. An algorithm for solving this problem, called a bucket sort , is given in Program gif .

program2218
Program: Bucket Sort

A bucket sort works as follows: An array of m counters, or buckets , is used. Each of the counters is set initially to zero. Then, a pass is made through the input array, during which the buckets are used to keep a count of the number of occurrences of each value between 0 and m-1. Finally, the sorted result is produced by first placing the required number of zeroes in the array, then the required number of ones, followed by the twos, and so on, up to m-1.

The analysis of the running time of Program gif is summarized in Table gif . Clearly, the worst-case running time of the first loop (lines 7-8) is O(m) and that of the second loop (lines 9-10) is O(n).

Table: Computing the running time of Program gif
 | time
statement | cursory analysis | careful analysis
7-8 | O(m) | O(m)
9-10 | O(n) | O(n)
11-13 | O(mn) | O(m+n)
TOTAL | O(mn) | O(m+n)

Consider nested loops on lines 11-13. Exactly m iterations of the outer loop are done--the number of iterations of the outer loop is fixed. But the number of iterations of the inner loop depends on bucket [j]--the value of the counter. Since there are n numbers in the input array, in the worst case a counter may have the value n. Therefore, the running time of lines 11-13 is O(mn) and this running time dominates all the others, so the running time of Program gif is O(mn). (This is the cursory analysis column of Table gif ).

Unfortunately, this time our analysis has not produced a tight bound. To see why this is the case, we must consider the operation of Program gif more carefully. In particular, since we are sorting n items, the final answer will only contain n items. Therefore, line 13 will be executed exactly n times--not mn times as the cursory result suggests.

Consider the inner loop at line 12. During the [image] iteration of the outer loop, the inner loop does [image] iterations. Therefore, the conditional test at line 12b is done [image] times. Therefore, the total number of times the conditional test is done is

So, the running time of lines 11-13 is O(m+n) and therefore running time of Program gif is O(m+n). (This is the careful analysis column of Table gif ).


Reality Check

``Asymptotic analysis is nice in theory,'' you say, ``but of what practical value is it when I don't know what c and [image] are?'' Fallacies gif and gif showed us that if we have two programs, A and B, that solve a given problem, whose running times are [image] and [image] say, we cannot conclude in general that we should use algorithm A rather than algorithm B to solve a particular instance of the problem. Even if the bounds are both known to be tight, we still don't have enough information. What we do know for sure is that eventually, for large enough n, program A is the better choice.

In practice we need not be so conservative. It is almost always the right choice to select program A. To see why this is the case, consider the times shown in Table gif . This table shows the running times computed for a very conservative scenario. We assume that the constant of proportionality, c, is one cycle of a 100 MHz clock. This table shows the running times we can expect even if only one instruction is done for each element of the input.

Table: Actual lower bounds assuming a 100 Mhz clock, [image] and
n=1 | n=8


Checking Your Analysis

Having made an asymptotic analysis of the running time of an algorithm, how can you verify that the implementation of the algorithm performs as predicted by the analysis? The only practical way to do this is to conduct an experiment--write out the algorithm in the form of a computer program, compile and execute the program, and measure its actual running time for various values of the parameter, n say, used to characterize the size of the problem.

However, several difficulties immediately arise:

  • How do you compare the results of the analysis which, by definition, only applies asymptotically, i.e., as n gets arbitrarily large, with the actual running time of a program which, of necessity, must be measured for fixed and finite values of n?
  • How do you explain it when the results of your analysis do not agree with the observed behavior of the program?

Suppose you have conducted an experiment in which you measured the actual running time of a program, T(n), for a number of different values of n. Furthermore, suppose that on the basis of an analysis of the algorithm you have concluded that the worst-case running time of the program is O(f(n)). How do you tell from the measurements made that the program behaves as predicted?

One way to do this follows directly from the definition of big oh: . This suggests that we should compute the ratio T(n)/f(n) for each of value of n in the experiment and observe how the ratio behaves as n increases. If this ratio diverges, then f(n) is probably too small; if this ratio converges to zero, then f(n) is probably too big; and if the ratio converges to a constant, then the analysis is probably correct.

What if f(n) turns out to large? There are several possibilities:

  • The function f(n) is not a tight bound. I.e., the analysis is still correct, but the bound is not the tightest bound possible.
  • The analysis was for the worst case but the worst case did not arise in the set of experiments conducted.
  • A mistake was made, and the analysis is wrong.


Exercises

  1. Consider the function [image]. Using Definition gif show that [image].
  2. Consider the function [image]. Using Definition gif show that [image].
  3. Consider the functions [image] and [image]. Using Theorem gif show that [image].
  4. Consider the functions [image] and [image]. Using Theorem gif show that [image].
  5. For each pair of functions, f(n) and g(n), in the following table, indicate if f(n)=O(g(n)) and if g(n)=O(f(n)).

    f(n) g(n)
    10n

  6. Show that the Fibonacci numbers (see Equation gif ) satisfy the identities

    for [image].

  7. Prove each of the following formulas:
  8. Show that [image], where [image] and [image].
  9. Show that [image].
  10. Solve each of the following recurrences:
  11. Derive tight, big oh expressions for the running times of sum.c,horner.c,factorial.c,findmax.c,geometric.c,geometric2.c,power.c,geometric3.c.
  12. Consider the C++ program fragments given below. Assume that n, m and k are unsigned ints and that the functions e, f, g and h have the following characteristics:
    • The worst case running time for e(n,m,k) is O(1) and it returns a value between 1 and (n+m+k).
    • The worst case running time for f(n,m,k) is O(n+m).
    • The worst case running time for g(n,m,k) is O(m+k).
    • The worst case running time for h(n,m,k) is O(n+k).
    Determine a tight, big oh expression for the worst-case running time of each of the following program fragments:
    1. f (n, 10, 0);
g (n, m, k);
h (n, m, 1000000);
    2. for (unsigned int i = 0; i < n; ++i)
    f (n, m, k);
    3. for (unsigned int i = 0; i < e (n, 10, 100); ++i)
    f (n, 10, 0);
    4. for (unsigned int i = 0; i < e (n, m, k); ++i)
    f (n, 10, 0);
    5. for (unsigned int i = 0; i < n; ++i)
    for (unsigned int j = i; j < n; ++j)
        f (n, m, k);

  13.
    Consider the following C++ program fragment.
    What value does f compute?
    (Express your answer as a function of n).
    Give a tight, big oh expression for the worst-case
    running time of the function f.

    unsigned int f (unsigned int n)
{
    unsigned int sum = 0;
    for (unsigned int i = 1; i <= n; ++i)
        sum = sum + i;
    return sum;
}
  14.
    Consider the following C++ program fragment.
    (The function f is given in Exercise   gif ).
    What value does g compute?
    (Express your answer as a function of n).
    Give a tight, big oh expression for the worst-case
    running time of the function g.

    unsigned int g (unsigned int n)
{
    unsigned int sum = 0;
    for (unsigned int i = 1; i <= n; ++i)
        sum = sum + i + f (n);
    return sum;
}
  15.
    Consider the following C++ program fragment.
    (The function f is given in Exercise   gif
    and the function g is given in Exercise   gif ).
    What value does h compute?
    (Express your answer as a function of n).
    Give a tight, big oh expression for the worst-case
    running time of the function h.

    unsigned int h (unsigned int n)
    { return f (n) + g (n); }

  next    up    previous    contents    index

  Bruno
Copyright © 1997 by Bruno R. Preiss, P.Eng.  All rights reserved.


Projects

  1. Write a C++ function that takes a single integer argument n and has a worst-case running time of O(n).
  2. Write a C++ function that takes a single integer argument n and has a worst-case running time of [image].
  3. Write a C++ function that takes two integer arguments n and k and has a worst-case running time of [image].
  4. Write a C++ function that takes a single integer argument n and has a worst-case running time of [image].
  5. Write a C++ function that takes a single integer argument n and has a worst-case running time of [image].
  6. Write a C++ function that takes a single integer argument n and has a worst-case running time of [image].
  7. The generalized Fibonacci numbers of order [image] are given by

    Write both recursive and non-recursive functions that compute [image]. Measure the running times of your algorithms for various values of k and n.


Foundational Data Structures

In this book we consider a variety of abstract data types (ADTs) , including stacks, queues, deques, ordered lists, sorted lists, hash and scatter tables, trees, priority queues, sets and graphs. In just about every case, we have the option of implementing the ADT using an array or using a some kind of linked data structure.

Because they are the base upon which almost all of the ADTs are built, we call the array and the linked list the foundational data structures . It is important to understand that we do not view the array or the linked list as ADTs, but rather as alternatives for the implementation of ADTs.

In this chapter we discuss in detail the implementation of two classes, Array<T> and LinkedList<T>, that embody the foundational data structures. It is important to become familiar with these classes, as they are used extensively throughout the remainder of the book.

  • Dynamic Arrays
  • Singly-Linked Lists
  • Multi-Dimensional Arrays
  • Exercises
  • Projects


Online Course Materials

Additional material supporting this book can be found on the world-wide web at the URL:

http://www.pads.uwaterloo.ca/Bruno.Preiss/books/opus4
In particular, you will find there the source code for all the program fragments in this book as well as an errata list.

Waterloo, Canada


Dynamic Arrays

Probably the most common way to aggregate data is to use an array. While the C++ programming language does indeed provide built-in support for arrays, that support is not without its shortcomings. Arrays in C++ are not first-class data types. There is no such thing as an array-valued expression. Consequently, you cannot use an array as an actual value parameter of a function; you cannot return an array value from a function; you cannot assign one array to another. (Of course, you can do all of these things with a pointer to an array). In addition, array subscripts range from zero to N-1, where N is the array size, and there is no bounds checking of array subscript expressions. And finally, the size of an array is static and fixed at compile time, unless dynamic memory allocation is explicitly used by the programmer.

Some of these characteristics of arrays are due in part to the fact that in C++, given a pointer, T* ptr, to some type, T, it is not possible to tell, just from the pointer itself, whether it points to a single instance of a variable of type T or to an array of variables of type T. Furthermore, even if we know that the pointer points to an array, we cannot determine the actual number of elements in that array.

It is primarily to address these deficiencies that we introduce the Array object which is implemented as a generic class. Figure gif illustrates the Array object is represented in the memory of the computer. Two structures are used. The first is a structure which comprises three fields--data, base and length. The member variable data is a pointer to the array data. Variables base and length are used in the array subscript calculation. The second structure comprises contiguous memory locations which hold the array elements. In the implementation given below, this second structure is allocated dynamically.

figure2462
Figure: Memory Representation of Array Objects

The C++ declaration of the Array<T> class template is given in Program gif . The Array<T> class has three protected member variables, data, base and length, constructors, destructor, and various member functions. The number of member functions has been kept to the bare minimum in this example--in the ``real world'' you can expect that such a class would contain many more useful member functions.

On the basis of Program gif , we can now calculate the total storage required to represent Array<T> objects. Let S(n) be the total storage (memory) needed to represent an Array<T> object which includes n array elements of type T. S(n) is given by

where the function [image] is the number of bytes used for the memory representation of an instance of an object of type X.

In C++, the sizes of the basic (built-in) data types are fixed constants. So too are the sizes of all pointers. Hence, [image] and [image]. Therefore,

Unfortunately, since Array<T> is a generic class, we have no a priori knowledge of the amount of storage used by an object of type T. However, if we assume that the amount of storage used by an object of type T is a fixed constant, then S(n)=O(n).

program2631
Program: Array<T> Class Definition

  • Default Constructor
  • Array Constructor
  • Copy Constructor
  • Destructor
  • Array Member Functions
  • Array Subscripting Operator
  • Resizing an Array


Default Constructor

The default constructor for an object of class T is the function T::T(). I.e., it is the constructor which takes no arguments. The definition of the Array<T> default destructor is given in Program gif . The default constructor allocates a zero-length array of elements of type T and sets the fields base and length to zero. Clearly, the running time of this routine is O(1).

program2665
Program: Array<T> Class Default Constructor Definition


Array Constructor

In this section we consider the Array<T> class constructor which takes two numeric arguments of type unsigned int. The definition of this constructor is given in Program gif . Given argument values m and n, the constructor first allocates an array of n elements of type T using operator new, and then sets the length field to n and the base field to m. The running time is a constant plus the time do the array allocation.

program2682
Program: Array<T> Class Constructor Definition

In C++, when an array is allocated, two things happen. First, memory is allocated for the array from the free store , known as the heap . Second, each element of the array is initialized by calling its default constructor.

For now, we shall assume that the first step in the dynamic allocation--the allocation of space in the heap--takes a constant amount of time. In fact, when we look at how the heap is implemented we will see that this assumption is does not always hold (see Chapter gif ). Suppose that the running time of the default constructor for objects of type T is given by [image]. Then, the time to initialize an array of size n is [image].

There is a special case to consider: The behavior of the default constructor for the C++ built-in types (int, char, ..., and pointers thereto) is to do nothing! So, if T is a built-in type, [image]. In this case, the running time for the Array<T> constructor is simply T(n)=O(1), regardless of the value n.


Copy Constructor

The copy constructor for an object of class T is the function T::T(T const&). I.e., it is the constructor for objects of class T which takes as its single argument a reference to another object of class T. This constructor builds a copy of the object passed by reference--hence the name copy constructor.

Copy constructors play a crucial rôle in C++ programs. For example, the copy constructor is called automatically by the compiler to pass the value of an actual parameter used in a function call to the formal parameter used in the function definition. Similarly, the copy constructor is called automatically by the compiler to pass the return value from a function back to the caller.

Program gif shows a simple, though perhaps somewhat naıve implementation for the copy constructor of Array<T> class objects. To determine its running time, we need to consider carefully the execution of this function.

program2714
Program: Array<T> Class Copy Constructor Definition

First, the required amount of memory is allocated by operator new. As discussed above, this involves finding space in the heap, which we assume takes constant time; and then initializing the elements of the array using the default constructor for objects of type T, which takes [image]. Next, the length field is set to the correct value, which takes constant time.

Finally, the body of the Array<T> constructor is a loop which copies one-by-one the elements of the input array to the newly allocated array. How this copy will actually be done depends on the type T. We shall assume that the running time of this assignment is the same as that of the copy constructor for objects of type T. gif Then, the running time of the main loop of the Array<T> constructor is [image] where [image] is the time taken by the copy constructor for objects of type T.

Altogether, the running time of the copy constructor for Array<T> is [image], where n is the size of the array being copied. In the case where T is one of the built-in types, [image] and [image], which gives the simple and obvious result, T(n)=O(n).


Destructor

Program gif shows the definition of the Array<T> destructor. The destructor simply invokes operator delete to deallocate the storage used by the array elements. In C++ two things happen when an array is deallocated. First, the destructor for each element of the array is called one-by-one. Second, the memory space used by the array is returned to the free store (heap).

program2737
Program: Array<T> Class Destructor Definition

If the running time of the destructor for an object of type T is [image], and assuming the time to return memory to the free store is a constant, the running time for the Array<T> destructor is [image]. In C++ the destructor default behavior of the destructor for a built-in types is to do nothing. Therefore, in the case where T is a built-in type, T(n)=O(1).


Array Member Functions

Program gif defines three Array<T> class member functions. The three functions Data, Base and Length provide a means for the user to inspect the contents of the Array<T> object.

program2756
Program: Array<T> Class Accessor Function Definitions

The functions Data, Base and Length, are const member functions . As such, they provide read-only access to the contents of the Array<T> object. Such functions are sometimes called member variable accessors . Clearly, the running times of each of these functions is a constant. I.e., T(n)=O(1).


Array Subscripting Operator

Program gif defines two functions, each of which overloads the array subscripting operation. The purpose of overloading operator[] is to allow arbitrary subscript ranges and to provide array bounds-checking. The array subscripting operator, operator[] is provided in two forms. One which operates on a const array object and returns a const reference to an object of type T; and another which does the same for non-const objects. The code in both instances is identical. The position argument is checked and then a reference to the selected item is returned. In the event of an array-bounds error, an outofrange gif exception is thrown.

We will assume that in a bug-free program there will never be an array-bounds error. Therefore, the running time of operator[] is a constant. I.e., T(n)=O(1). This is, after all, the sole reason for using an array--the time it takes to access the [image] element of the array is a constant!

program2795
Program: Array<T> Class Subscripting Function Definitions


Resizing an Array

The definitions the SetBase and SetLength member functions of the Array<T> class are given in Program gif . The SetBase function simply modifies the base field as required. The SetLength function provides a means to change the size of an array at run time. This function can be used both to increase and to decrease the size of an array.

program2812
Program: Array<T> Class SetBase and SetLength Member Function Definitions

The running time of this algorithm depends on two parameters. If n is the original size of the array and m is the new size of the array, then the running time of Resize is [image]. This is because the program first allocates and initializes a new array of size m; then it copies at most [image] elements from the old array to the new array; and then it deletes the old array. And in the case where T is a built-in C++ data type, this simplifies to [image].


Singly-Linked Lists

The singly-linked list is the most basic of all the pointer-based data structures. A singly-linked list is simply a sequence of dynamically allocated storage elements, each containing a pointer to its successor. Despite this obvious simplicity, there are myriad implementation variations. Figure gif shows several of the most common singly-linked list variants.

figure2829
Figure: Singly-Linked List Variations

The basic singly-linked list is shown in Figure gif (a). Each element of the list contains a pointer to its successor; the last element contains a null pointer. A pointer to the first element of the list, labeled head in Figure gif (a), is used to keep track of the list.

The basic singly-linked list is inefficient in those cases when we wish to add elements to both ends of the list. While it is easy to add elements at the head of the list, to add elements at the other end (the tail ) we need to locate the last element. If the basic basic singly-linked list is used, the entire list needs to be traversed in order to find its tail.

Figure gif (b) shows a way in which to make adding elements to the tail of a list more efficient. The solution is to keep a second pointer, tail , which points to the last element of the list. Of course, this time efficiency comes at the cost of the additional space used to store the tail pointer.

The singly-linked lists labeled (c) and (d) in Figure gif illustrate two common programming tricks. The list (c) has an extra element at the head of the list called a sentinel . This element is never used to hold data and it is always present. The principal advantage of using a sentinel is that it simplifies the programming of certain operations. E.g., since there is always a sentinel standing guard, we never need to modify the head pointer. Of course, the disadvantage of a sentinel such as that shown in (c) is that extra space is required, and the sentinel needs to be created when the list is initialized.

The list (c) is also a circular list . Instead of using a null pointer to demarcate the end of the list, the pointer in the last element points to the sentinel. The advantage of this programming trick is that insertion at the head of the list, insertion at the tail of the list, and insertion at an arbitrary position of the list are all identical operations.

Figure gif (d) shows a variation of a singly-linked list using a sentinel in which instead of keeping a pointer to the sentinel, the sentinel itself serves as the handle for the list. This variant eliminates the need to allocate storage for the sentinel separately.

Of course, it is also possible to make a circular, singly-linked list that does not use a sentinel. Figure gif (e) shows a variation in which a single pointer is used to keep track of the list, but this time the pointer, tail, points to the last element of the list. Since the list is circular in this case, the first element follows the last element of the list. Therefore, it is relatively simple to insert both at the head and at the tail of this list. This variation minimizes the storage required, at the expense of a little extra time for certain operations.

Figure gif illustrates how the empty list (i.e., the list containing no list elements) is represented for each of the variations given in Figure gif . Notice that the sentinel is always present in those list variants which use it. On the other hand, in the list variants which do not use a sentinel, null pointers are used to indicate the empty list.

figure3089
Figure: Empty Singly-Linked Lists

The list variant (b) introduces a potential source of very subtle programming errors. A conservative programmer would insist that both the head and tail pointers must be null. However, a clever programmer might realize that it is sufficient require only that the head pointer be null in the case of an empty list. Since, the tail pointer is not used when the list is empty, its value may be left undefined. Of course, if that is the case, extreme care must be taken to ensure that the tail pointer is never used when the head pointer is null.

In the following sections, we will present the implementation details of a generic singly-linked list. We have chosen to present variation (b)--the one which uses a head and a tail pointer--since is supports append and prepend operations efficiently. While linked lists which use a sentinel do present some interesting time efficiencies, we have chosen not to use a sentinel because the use of a sentinel is essentially a programming trick. Also, from an object-oriented perspective, the use of a sentinel introduces some semantic difficulties. E.g., if each element of the list contains an object of type T, and we create a list to hold exactly n elements, it might be reasonable to expect that T's constructor is called only n times. However, when using a sentinel n+1 objects of type T are actually created--the extra one being the sentinel itself. Thus, the constructor of T objects is called n+1 times which may not be at all what the programmer expects.

  • An Implementation
  • List Elements
  • Default Constructor
  • Destructor and Purge Member Function
  • Accessors
  • First and Last Functions
  • Prepend
  • Append
  • Copy Constructor and Assignment Operator
  • Extract
  • InsertAfter and InsertBefore


An Implementation

Figure gif illustrates the the singly-linked list scheme we have chosen to implement. Two structures are used. The elements of the list are represented using a structure which comprises two fields--datum and next. The former holds the a single data item; the latter is a pointer to the next array element. The main structure also comprises two fields-head and tail. Which contain pointers to the first and last list elements, respectively.

figure3255
Figure: Memory Representation of Linked List Objects

Program gif gives the declaration of the LinkedList<T> and ListElement<T> class templates. The ListElement<T> class is used to represent the elements of a linked list. It has two private member variables, datum and next, a private constructor, and two public accessor functions.

program3445
Program: LinkedList<T> and ListElement<T> Class Definitions

A LinkedList<T> class object has two protected member variables, head and tail, several constructors, a destructor, and various other member functions.

We can calculate the total storage required, S(n), to hold a linked list of n items from the class definitions given in Program gif as follows:

Since LinkedList<T> and ListElement<T> are generic classes, we have no a priori knowledge of the amount of storage used by an object of type T. However, it is often reasonable to assume that [image]. gif And since a pointer requires a constant amount of space, [image]. If we assume that the amount of storage used by an object of type T is O(1), then S(n)=O(n).


Contents

  • Colophon
  • Dedication
  • Preface
    • Goals
    • Approach
    • Outline
    • Suggested Course Outline
    • Online Course Materials
  • Introduction
    • What This Book Is About
    • Object-Oriented Design
      • Abstraction
      • Encapsulation
    • Object Hierarchies and Design Patterns
      • Containers
      • Iterators
      • Visitors
      • Adapters
      • Singletons
    • The Features of C++ You Need to Know
      • Variables
      • Parameter Passing
      • Pointers
      • Classes and Objects
      • Inheritance
      • Other Features
    • How This Book Is Organized
      • Models and Asymptotic Analysis
      • Foundational Data Structures
      • Abstract Data Types and the Class Hierarchy
      • Data Structures
      • Algorithms
  • Algorithm Analysis
    • A Detailed Model of the Computer
      • The Basic Axioms
      • A Simple Example-Arithmetic Series Summation
      • Array Subscripting Operations
      • Another Example-Horner's Rule
      • Analyzing Recursive Functions
        • Solving Recurrence Relations-Repeated Substitution
      • Yet Another Example-Finding the Largest Element of an Array
      • Average Running Times
      • About Harmonic Numbers
      • Best-Case and Worst-Case Running Times
      • The Last Axiom
    • A Simplified Model of the Computer
      • An Example-Geometric Series Summation
      • About Arithmetic Series Summation
      • Example-Geometric Series Summation Again
      • About Geometric Series Summation
      • Example-Computing Powers
      • Example-Geometric Series Summation Yet Again
    • Exercises
    • Projects
  • Asymptotic Notation
    • An Asymptotic Upper Bound-Big Oh
      • A Simple Example
      • Big Oh Fallacies and Pitfalls
      • Properties of Big Oh
      • About Polynomials
      • About Logarithms
      • Tight Big Oh Bounds
      • More Big Oh Fallacies and Pitfalls
      • Conventions for Writing Big Oh Expressions
    • An Asymptotic Lower Bound-Omega
      • A Simple Example
      • About Polynomials Again
    • More Notation-Theta and Little Oh
    • Asymptotic Analysis of Algorithms
      • Rules For Big Oh Analysis of Running Time
      • Example-Prefix Sums
      • Example-Fibonacci Numbers
      • Example-Bucket Sort
      • Reality Check
      • Checking Your Analysis
    • Exercises
    • Projects
  • Foundational Data Structures
    • Dynamic Arrays
      • Default Constructor
      • Array Constructor
      • Copy Constructor
      • Destructor
      • Array Member Functions
      • Array Subscripting Operator
      • Resizing an Array
    • Singly-Linked Lists
      • An Implementation
      • List Elements
      • Default Constructor
      • Destructor and Purge Member Function
      • Accessors
      • First and Last Functions
      • Prepend
      • Append
      • Copy Constructor and Assignment Operator
      • Extract
      • InsertAfter and InsertBefore
    • Multi-Dimensional Arrays
      • Array Subscript Calculations
      • Two-Dimensional Array Implementation
      • Multi-Dimensional Subscripting in C++
      • Canonical Matrix Multiplication
    • Exercises
    • Projects
  • Data Types and Abstraction
    • Abstract Data Types
    • Design Patterns
      • Class Hierarchy
      • Objects
        • Implementation
      • The NullObject Singleton Class
        • Implementation
      • Object Wrappers for the Built-In Types
        • Implementation
      • Containers
      • Visitors
        • The IsDone Member Function
        • Container Class Default Put Member Function
      • Iterators
      • The NullIterator Class
      • Direct vs. Indirect Containment
      • Ownership of Contained Objects
      • Associations
        • Implementation
      • Searchable Containers
    • Exercises
    • Projects
  • Stacks, Queues and Deques
    • Stacks
      • Array Implementation
        • Member Variables
        • Constructor and Destructor
        • Push, Pop, and Top Member Functions
        • The Accept Member Function
        • Iterator
      • Linked List Implementation
        • Member Variables
        • Constructor and Destructor
        • Push, Pop, and Top Member Functions
        • The Accept Member Function
        • Iterator
      • Applications
        • Evaluating Postfix Expressions
        • Implementation
    • Queues
      • Array Implementation
        • Member Variables
        • Constructor and Destructor
        • Head, Enqueue, and Dequeue Member Functions
      • Linked List Implementation
        • Member Variables
        • Constructor and Destructor
        • Head, Enqueue and Dequeue Member Functions
      • Applications
        • Implementation
    • Deques
      • Array Implementation
        • Tail, EnqueueHead, and DequeueTail Member Functions
      • Linked List Implementation
        • Tail, EnqueueHead, and DequeueTail Member Functions
      • Doubly-Linked and Circular Lists
    • Exercises
    • Projects
  • Ordered Lists and Sorted Lists
    • Ordered Lists
      • Array Implementation
        • Member Variables
        • Inserting and Accessing Items in a List
        • Finding Items in a List
        • Removing Items from a List
        • Positions of Items in a List
        • Finding the Position of an Item and Accessing by Position
        • Inserting an Item at an Arbitrary Position
        • Removing Arbitrary Items by Position
      • Linked List Implementation
        • Member Variables
        • Inserting and Accessing Items in a List
        • Finding Items in a List
        • Removing Items from a List
        • Positions of Items in a List
        • Finding the Position of an Item and Accessing by Position
        • Inserting an Item at an Arbitrary Position
        • Removing Arbitrary Items by Position
      • Performance Comparison: ListAsArray vs. ListAsLinkedList
      • Applications
    • Sorted Lists
      • Array Implementation
        • Inserting Items in a Sorted List
        • Locating Items in an Array-Binary Search
        • Finding Items in a Sorted List
        • Removing Items from a List
      • Linked List Implementation
        • Inserting Items in a Sorted List
        • Other Operations on Sorted Lists
      • Performance Comparison: SortedListAsArray vs. SortedListAsList
      • Applications
        • Implementation
        • Analysis
    • Exercises
    • Projects
  • Hashing, Hash Tables and Scatter Tables
    • Hashing-The Basic Idea
      • Example
      • Keys and Hash Functions
        • Avoiding Collisions
        • Spreading Keys Evenly
        • Ease of Computation
    • Hashing Methods
      • Division Method
      • Middle Square Method
      • Multiplication Method
      • Fibonacci Hashing
    • Hash Function Implementations
      • Integral Keys
      • Floating-Point Keys
      • Character String Keys
      • Hashing Objects
      • Hashing Containers
      • Using Associations
    • Hash Tables
      • Separate Chaining
        • Implementation
        • Constructor and Destructor
        • Inserting and Removing Items
        • Finding an Item
      • Average Case Analysis
    • Scatter Tables
      • Chained Scatter Table
        • Implementation
        • Constructors and Destructor
        • Inserting and Finding an Item
        • Removing Items
        • Worst-Case Running Time
      • Average Case Analysis
    • Scatter Table using Open Addressing
      • Linear Probing
      • Quadratic Probing
      • Double Hashing
      • Implementation
        • Constructors and Destructor
        • Inserting Items
        • Finding Items
        • Removing Items
      • Average Case Analysis
    • Applications
    • Exercises
    • Projects
  • Trees
    • Basics
      • Terminology
      • More Terminology
      • Alternate Representations for Trees
    • N-ary Trees
    • Binary Trees
    • Tree Traversals
      • Preorder Traversal
      • Postorder Traversal
      • Inorder Traversal
      • Breadth-First Traversal
    • Expression Trees
      • Infix Notation
      • Prefix Notation
      • Postfix Notation
    • Implementing Trees
      • Tree Traversals
        • Depth-First Traversal
        • Preorder, Inorder and Postorder Traversals
        • Breadth-First Traversal
        • Accept Member Function
      • Tree Iterators
        • Member Variables
        • Constructor and Reset Member Function
        • Operator Member Functions
      • General Trees
        • Member Variables
        • Member Functions
        • Constructor, Destructor, and Purge Member Function
        • Key and Subtree Member Functions
        • AttachSubtree and DetachSubtree Member Functions
      • N-ary Trees
        • Member Variables
        • Member Functions
        • Constructors
        • IsEmpty Member Function
        • Key, AttachKey and DetachKey Member Functions
        • Subtree, AttachSubtree and DetachSubtree Member Functions
      • Binary Trees
        • Member Variables
        • Constructors
        • Destructor and Purge Member Functions
      • Binary Tree Traversals
      • Comparing Trees
      • Applications
        • Implementation
    • Exercises
    • Projects
  • Search Trees
    • Basics
      • M-Way Search Trees
      • Binary Search Trees
    • Searching a Search Tree
      • Searching an M-way Tree
      • Searching a Binary Tree
    • Average Case Analysis
      • Successful Search
      • Solving The Recurrence-Telescoping
      • Unsuccessful Search
      • Traversing a Search Tree
    • Implementing Search Trees
      • Binary Search Trees
        • Member Variables
        • Find Member Function
        • FindMin Member Function
      • Inserting Items in a Binary Search Tree
        • Insert and AttachKey Member Functions
      • Removing Items from a Binary Search Tree
        • Withdraw and DetachKey Member Functions
    • AVL Search Trees
      • Implementing AVL Trees
        • Constructor
        • Height, AdjustHeight and BalanceFactor Member Functions
      • Inserting Items into an AVL Tree
        • Balancing AVL Trees
        • Single Rotations
        • Double Rotations
        • Implementation
      • Removing Items from an AVL Tree
    • M-Way Search Trees
      • Implementing M-Way Search Trees
        • Implementation
        • Member Functions
        • Inorder Traversal
      • Finding Items in an M-Way Search Tree
        • Linear Search
        • Binary Search
      • Inserting Items into an M-Way Search Tree
      • Removing Items from an M-Way Search Tree
    • B-Trees
      • Implementing B-Trees
        • Member Variables
        • Constructors
        • Private Member Functions
      • Inserting Items into a B-Tree
        • Implementation
        • Running Time Analysis
      • Removing Items from a B-Tree
    • Applications
    • Exercises
    • Projects
  • Heaps and Priority Queues
    • Basics
    • Binary Heaps
      • Complete Trees
        • Complete N-ary Trees
      • Implementation
        • Member Variables
        • Constructor, Destructor and Purge Member Functions
      • Putting Items into a Binary Heap
      • Removing Items from a Binary Heap
    • Leftist Heaps
      • Leftist Trees
      • Implementation
        • Member Variables
        • SwapContents Member Function
      • Merging Leftist Heaps
      • Putting Items into a Leftist Heap
      • Removing Items from a Leftist Heap
    • Binomial Queues
      • Binomial Trees
      • Binomial Queues
      • Implementation
        • Heap-Ordered Binomial Trees
        • Binomial Queues
        • Member Variables
        • AddTree and RemoveTree
        • FindMinTree and FindMin Member Functions
      • Merging Binomial Queues
      • Putting Items into a Binomial Queue
      • Removing an Item from a Binomial Queue
    • Applications
      • Discrete Event Simulation
      • Implementation
    • Exercises
    • Projects
  • Sets, Multisets and Partitions
    • Basics
      • Implementing Sets
    • Array and Bit-Vector Sets
      • Basic Operations
      • Union, Intersection and Difference
      • Comparing Sets
      • Bit-Vector Sets
        • Basic Operations
        • Union, Intersection and Difference
    • Multisets
      • Array Implementation
        • Basic Operations
        • Union, Intersection and Difference
      • Linked List Implementation
        • Union
        • Intersection
    • Partitions
      • Representing Partitions
      • Implementing a Partition using a Forest
        • Implementation
        • Constructors and Destructor
        • Find and Join Member Functions
      • Collapsing Find
      • Union by Size
      • Union by Height or Rank
    • Applications
    • Exercises
    • Projects
  • Dynamic Storage Allocation: The Other Kind of Heap
    • Basics
      • C++ Magic
        • Working with Multiple Storage Pools
      • The Heap
    • Singly Linked Free Storage
      • Implementation
        • Constructor and Destructor
        • Acquiring an Area
        • Releasing an Area
    • Doubly Linked Free Storage
      • Implementation
        • Constructor and Destructor
        • Releasing an Area
        • Acquiring an Area
    • Buddy System for Storage Management
      • Implementation
        • Constructor and Destructor
        • Acquiring an Area
        • Releasing an Area
    • Applications
      • Implementation
    • Exercises
    • Projects
  • Algorithmic Patterns and Problem Solvers
    • Brute-Force and Greedy Algorithms
      • Example-Counting Change
        • Brute-Force Algorithm
        • Greedy Algorithm
      • Example-0/1 Knapsack Problem
    • Backtracking Algorithms
      • Example-Balancing Scales
      • Representing the Solution Space
      • Abstract Backtracking Solvers
        • Depth-First Solver
        • Breadth-First Solver
      • Branch-and-Bound Solvers
        • Depth-First, Branch-and-Bound Solver
      • Example-0/1 Knapsack Problem Again
    • Top-Down Algorithms: Divide-and-Conquer
      • Example-Binary Search
      • Example-Computing Fibonacci Numbers
      • Example-Merge Sorting
      • Running Time of Divide-and-Conquer Algorithms
        • Case 1 ( [image])
        • Case 2 ( [image])
        • Case 3 ( [image])
        • Summary
      • Example-Matrix Multiplication
    • Bottom-Up Algorithms: Dynamic
      Programming
      • Example-Generalized Fibonacci Numbers
      • Example-Computing Binomial Coefficients
      • Application: Typesetting Problem
        • Example
        • Implementation
    • Randomized Algorithms
      • Generating Random Numbers
        • The Minimal Standard Random Number Generator
        • Implementation
      • Random Variables
        • Implementation
      • Monte Carlo Methods
        • Example-Computing
      • Simulated Annealing
        • Example-Balancing Scales
    • Exercises
    • Projects
  • Sorting Algorithms and Sorters
    • Basics
    • Sorting and Sorters
      • Sorter Class Hierarchy
    • Insertion Sorting
      • Straight Insertion Sort
        • Implementation
      • Average Running Time
      • Binary Insertion Sort
    • Exchange Sorting
      • Bubble Sort
      • Quicksort
        • Implementation
      • Running Time Analysis
        • Worst-Case Running Time
        • Best-Case Running Time
      • Average Running Time
      • Selecting the Pivot
    • Selection Sorting
      • Straight Selection Sorting
        • Implementation
      • Sorting with a Heap
        • Implementation
      • Building the Heap
        • Running Time Analysis
        • The Sorting Phase
    • Merge Sorting
      • Implementation
      • Merging
      • Two-Way Merge Sorting
      • Running Time Analysis
    • A Lower Bound on Sorting
    • Distribution Sorting
      • Bucket Sort
        • Implementation
      • Radix Sort
        • Implementation
    • Performance Data
    • Exercises
    • Projects
  • Graphs and Graph Algorithms
    • Basics
      • Directed Graphs
      • Terminology
      • More Terminology
      • Directed Acyclic Graphs
      • Undirected Graphs
      • Terminology
      • Labeled Graphs
      • Representing Graphs
        • Adjacency Matrices
        • Sparse vs. Dense Graphs
        • Adjacency Lists
    • Implementing Graphs
      • Implementing Vertices
      • Implementing Edges
      • Abstract Graphs and Digraphs
        • Accessors and Mutators
        • Iterators
        • Graph Traversals
      • Implementing Undirected Graphs
        • Using Adjacency Matrices
        • Using Adjacency Lists
      • Edge-Weighted and Vertex-Weighted Graphs
      • Comparison of Graph Representations
        • Space Comparison
        • Time Comparison
    • Graph Traversals
      • Depth-First Traversal
        • Implementation
        • Running Time Analysis
      • Breadth-First Traversal
        • Implementation
        • Running Time Analysis
      • Topological Sort
        • Implementation
        • Running Time Analysis
      • Graph Traversal Applications:
        Testing for Cycles and Connectedness
        • Connectedness of an Undirected Graph
        • Connectedness of a Directed Graph
        • Testing for Cycles in a Directed Graph
    • Shortest-Path Algorithms
      • Single-Source Shortest Path
        • Dijkstra's Algorithm
        • Data Structures for Dijkstra's Algorithm
        • Implementation
        • Running Time Analysis
      • All-Pairs Source Shortest Path
        • Floyd's Algorithm
        • Implementation
        • Running Time Analysis
    • Minimum-Cost Spanning Trees
      • Constructing Spanning Trees
      • Minimum-Cost Spanning Trees
      • Prim's Algorithm
        • Implementation
      • Kruskal's Algorithm
        • Implementation
        • Running Time Analysis
    • Application: Critical Path Analysis
      • Implementation
    • Exercises
    • Projects
  • C++ and Object-Oriented Programming
    • Variables, Pointers and References
      • Pointers Are Variables
        • Dereferencing Pointers
      • References are Not Variables
    • Parameter Passing
      • Pass By Value
      • Pass By Reference
        • The Trade-off
        • Constant Parameters
    • Objects and Classes
      • Member Variables and Member Functions
      • Constructors and Destructors
        • Default Constructor
        • Copy Constructor
        • The Copy Constructor, Parameter Passing and Function Return Values
        • Destructors
      • Accessors and Mutators
        • Mutators
        • Member Access Control
    • Inheritance and Polymorphism
      • Derivation and Inheritance
        • Derivation and Access Control
      • Polymorphism
        • Virtual Member Functions
        • Abstract Classes and Concrete Classes
        • Algorithmic Abstraction
      • Multiple Inheritance
      • Run-Time Type Information and Casts
    • Templates
    • Exceptions
  • Class Hierarchy Diagrams
  • Character Codes
  • References
  • Index

next up previous index


List Elements

The definitions of the member functions of the ListElement<T> class are given in Program gif . Altogether, there are three functions--a constructor and two accessors.

program3520
Program: ListElement<T> Class Member Function Definitions

The constructor simply initializes the member variable to the passed values. The running time of the constructor is [image], where [image] is the running time of the copy constructor for objects of class T. Assigning a value to the next field takes a constant amount of time.

The two accessor functions, Datum and Next, simply return the values of the corresponding member variables. Clearly, the running times of each of these functions is O(1). Note that if Datum had been defined as returning a T, rather than a reference to T, the assumption of constant running time would not be valid.


Default Constructor

The code for the default LinkedList<T> constructor is given in Program gif . This routine simply constructs the representation of an empty linked list as shown in Figure gif (b). I.e., both the head and tail pointers are set to zero. The running time of the default constructor is clearly constant. I.e., T(n)=O(1).

program3546
Program: LinkedList<T> Class Default Constructor Definition


Destructor and Purge Member Function

Program gif gives the code for the destructor and the Purge member function of the LinkedList<T> class. In the general case, when the destructor is invoked, we cannot expect the list to be empty. Therefore, it is the responsibility of the destructor to first release any ListElement<T>s which have been allocated and linked into the list. And this is precisely what the Purge member function does.

program3560
Program: LinkedList<T> Class Destructor and Purge Member Function Definitions

The main loop of the Purge function simply traverses all the elements of linked list, deleting each of them one-by-one. Note that the body of the loop has been carefully written so as to determine the new value of head, before deleting the current list element. E.g., the following loop body is the wrong way to do it:

while (head != 0)
{
    delete head;
    head = head->next; // Wrong.  Don't do this.
}
The problem with the code fragment above is that the head pointer is dereferenced after the object to which it points has been deleted. While this code might actually work fortuitously for some compilers on some machines under some operating systems, it is invalid and illegal.


Accessors

Three LinkedList<T> class member variable accessors are defined in Program gif . The member functions Head and Tail provide read-only access to the corresponding data members of LinkedList<T>. The function IsEmpty returns a Boolean result which indicates whether the list is empty. Clearly, the running time of the of the accessors is O(1).

program3583
Program: LinkedList<T> Class Accessor Function Definitions


First and Last Functions

Two more LinkedList<T> class object accessors are defined in Program gif . The First member function returns a const reference to the first list element. Similarly, the Last member function returns a const reference to the last list element.

program3603
Program: LinkedList<T> Class First and Last Function Definitions

The code for both functions is almost identical. In the event that the list is empty, a domainerror gif exception is thrown. An exception is thrown because if the list is empty, it is not possible even to create a return value--there is no T object instance to which to refer!

We will assume that in a bug-free program, neither the First nor the Last functions will be called for an empty list. In that case, the running time of each of theses functions is constant. I.e., T(n)=O(1).


Prepend

To prepend an element to a linked list is to insert that element in front of the first element of the list. The prepended list element becomes the new head of the list. Program gif gives the algorithm for the Prepend member function of the LinkedList<T> class.

program3629
Program: LinkedList<T> Class Prepend Function Definition

The Prepend function first allocates a new ListElement<T>. Its datum field is initialized with the value to be prepended to the list, item; and the next field is made to point to the first element of the existing list by initializing it with the value of the current head pointer. If the list is initially empty, both head and tail are made to point at the new element. Otherwise, just head needs to be updated.

Note, operator new calls the constructor for objects of class ListElement<T>. In Section gif the running time of the constructor was determined to be [image]. And since the body of the Prepend function adds only a constant amount of work, the running time of the Prepend function is also [image]. If T is one of the built-in types, then its copy constructor does a constant amount of work, and the running time of Prepend simplifies to O(1).


Append

The Append function, the definition of which is given in Program gif , adds a new ListElement<T> at the tail-end of the list. The appended element becomes the new tail of the list.

program3659
Program: LinkedList<T> Class Append Function Definition

The Append function first allocates a new ListElement<T>. Its datum field is initialized with the value to be appended, and the next field is set to zero. If the list is initially empty, both head and tail are made to point at the new element. Otherwise, the new element is appended to the existing list, and the just tail pointer is updated.

The running time analysis of the Append function is essentially the same as for Prepend. I.e, the running time is [image]. And if T is one of the built-in types, the running time simplifies to O(1).


Copy Constructor and Assignment Operator

The code for the copy constructor and assignment operator (operator=) of the LinkedList<T> class is given in Program gif . These functions are similar in that they both build a copy of a given list. The copy constructor first initializes the member variables to represent the empty list. Then, it traverses the referenced list one-by-one, calling the Append function to append the items of the referenced list to the list begin constructed.

program3683
Program: LinkedList<T> Class Copy Constructor Definition

In Section gif the running time for the Append function was determined to be [image]. If the resulting list has n elements, the Append function will be called n times. Therefore, the running time of the copy constructor is [image]. If T is one of the built-in types, [image]. As a result, the running time of the copy constructor would be T(n)=O(n).

The assignment operator first calls Purge to make sure that the list to which new contents are being assigned is empty. It then builds a copy of the given list in the same way as discussed above for the copy constructor. The running time is equal to that of the Purge function plus that of the copy constructor.


Extract

In this section we consider the Extract member function of the LinkedList<T> class. The purpose of this function is to delete the specified element from the linked list.

program3702
Program: LinkedList<T> Class Extract Function Definition

The element to be deleted is identified by its value. The Extract function searches sequentially for the item to be deleted. In the absence of any a priori knowledge, we do not know in which list element the item to be deleted will be found. In fact, the specified item may not even appear in the list!

If we assume that the item to be deleted is in the list, and if we assume that there is an equal probability of finding it in each of the possible positions, then on average we will need to search half way through the list before the item to be deleted is found. In the worst case, the item will be found at the tail--assuming it is in the list.

If the item to be deleted does not appear in the list, the algorithm shown in Program gif throws a domainerror exception. A simpler alternative might be to do nothing--after all, if the item to be deleted is not in the list, then we are already done! However, attempting to delete an item which is not there, is more likely to indicate a logic error in the programming. It is for this reason that an exception is thrown.

In order to determine the running time of the Extract function, we first need to determine the time to find the element to be deleted. And to determine that, we need to know the running time for the comparison on line 8. Unfortunately, since LinkedList<T> is a generic class, we don't know in general what the running time of the comparison will be. So we need to introduce yet another variable to represent this unknown. Let [image] be the time required to determine if two objects of type T are equal.

If the item to be deleted is not in the list, then the running time of Program gif up to the point where it throws the exception (line 12) is [image], which simplifies to T(n)=O(n) if [image].

Now consider what happens if the item to be deleted is found in the list. In the worst-case the item to be deleted is at the tail. Thus, the running time to find the element is [image] in the worst case. Actually deleting the element from the list once it has been found is a short sequence of relatively straight-forward pointer manipulations. These manipulations can be done in constant time. Finally, the list element, having been unlinked from the list, is returned to the free store. So, the total running time is [image]. For a built-in type, [image] and [image], which gives the running time T(n)=O(n).


InsertAfter and InsertBefore

The functions InsertAfter and InsertBefore both take two arguments. The first is a pointer to an element of the linked list and the second is a reference to the item to be inserted into the list. The item is inserted either in front of or immediately following the indicated list element. Program gif gives the code for the InsertAfter and InsertBefore member functions.

program3729
Program: LinkedList<T> Class InsertAfter and InsertBefore Function Definitions

The InsertAfter member function is almost identical to Append. Whereas Append inserts an item after the tail, InsertAfter inserts an item after an arbitrary list element. Nevertheless, the running time of InsertAfter is identical to that of Append, i.e., it is [image].

To insert a new item before a given list element, it is necessary to traverse the linked list starting from the head to locate the list element that precedes the given list element. In the worst case, the given element is the at the tail of the list and the entire list needs to be traversed. Therefore, the running time of the InsertBefore member function is [image].


404 Not Found

Not Found

The requested URL /papers/published/1999/paper.html was not found on this server.

Apache/1.3.14 Server at www.brpreiss.com Port 80


 Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Degrees Up: Curriculum Vita

Bruno Richard Preiss:
Curriculum Vita

B.A.Sc., M.A.Sc., Ph.D., P.Eng.

January 26, 2002

  • Degrees
  • Diplomas
  • Scholarships and Awards
  • Employment record
  • Membership in Professional Organizations and Societies
  • Publications
    • Refereed Full Journal Papers
    • Submitted Journal Papers
    • Refereed Technical Notes
    • Patents
    • Books
    • Conference Papers (full paper refereed)
    • Conference Papers (abstract refereed)
    • Submitted Conference Papers
    • Technical Reports
    • Theses
  • Presentations, Seminars, Invited Talks
  • Research Grants
  • Teaching Record
  • Supervision of Students
    • Graduate Students
    • NSERC Undergraduate Summer Research Assistants/Co-op Students
    • UW Undergraduate Research Assistants
    • E&CE 499 Projects
    • Graduate Student Theses Supervised
  • Service to the University of Waterloo
  • Service to the Profession

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Sat Jan 26 21:21:33 EST 2002


Degrees

  1. Ph.D., 1987, University of Toronto, Department of Electrical Engineering. Dissertation title: Dataflow on a Queue Machine. Supervisor: Dr. V. Carl Hamacher.
  2. M.A.Sc., 1984, University of Toronto, Department of Electrical Engineering. Dissertation title: Design and Simulation of a Data-Flow Multiprocessor System. Supervisor: Dr. V. Carl Hamacher.
  3. B.A.Sc., 1982, University of Toronto, Engineering Science Division, (Electrical Engineering Option). Dissertation title: LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks. Supervisor: Dr. V. Carl Hamacher.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Patents

  1. B. R. Preiss and D. Wilk. Method, system and apparatus for provisioning services to subscribers in a telecommunications network, May 2001. Patent Application.
  2. B. R. Preiss and D. Wilk. Interactive voice response system and method, April 2001. Patent Application.
  3. M. Stumm, W. M. Snelgrove, B. R. Preiss, M. Frazer, and M. De Simone. Telecommunication architecture, October 2000. Patent Application.
  4. B. R. Preiss. Strategy for negotiation of telecommunication resources, March 2000. Patent Application.
  5. W. M. Snelgrove, M. Stumm, M. De Simone, and B. R. Preiss. Method and system for configuring communication resources, January 2000. Patent Application.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Books

  1. B. R. Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 2000. 635 pp. ISBN 0-471-34613-6.
  2. B. R. Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in Java. John Wiley & Sons, 1999. 162 pp. ISBN 0-471-36916-0.
  3. B. R. Preiss. Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 660 pp. ISBN 0471-24134-2.
  4. B. R. Preiss. Solutions Manual: Data Structures and Algorithms with Object-Oriented Design Patterns in C++. John Wiley & Sons, 1999. 160 pp. ISBN 0-471-32211-3.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Conference Papers (full paper refereed)

  1. B. R. Preiss. Design patterns for the data structures and algorithms course. In Proc. SIGCSE 1999 Technical Symposium, pages 95-99, New Orleans, LA, March 1999. Association for Computing Machinery, Inc.
  2. B. R. Preiss and C. K. W. Wan. The Parsimony project: A distributed simulation testbed in Java. In Proc. 1999 International Conference On Web-Based Modelling & Simulation, volume 31 of Simulation Series, pages 89-94, San Francisco, CA, January 1999. Society for Computer Simulation.
  3. B. R. Preiss and W. M. Loucks. Memory management techniques for Time Warp on a distributed memory machine. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 30-39, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.
  4. B. R. Preiss. An algorithm for speculative parallel execution of rendezvous-synchronized simulation. In Proc. 1995 Workshop on Parallel and Distributed Simulation, pages 11-20, Lake Placid, NY, June 1995. Institute of Electrical and Electronics Engineers, Inc.
  5. Y.-B. Lin, B. R. Preiss, W. M. Loucks, and E. D. Lazowska. Selecting the checkpoint interval in Time Warp parallel simulation. In Proc. 1993 Workshop on Parallel and Distributed Simulation, pages 3-10, San Diego, CA, May 1993. Institute of Electrical and Electronics Engineers, Inc.
  6. A. Rooks and B. R. Preiss. A unifying framework for distributed routing algorithms. In Proc. 1992 Int. Conf. on Parallel Processing, pages I-96-I-100, St. Charles, IL, August 1992. Penn. State University.
  7. B. R. Preiss, I. D. MacIntyre, and W. M. Loucks. On the trade-off between time and space in optimistic parallel discrete-event simulation. In Proc. 1992 Workshop on Parallel and Distributed Simulation, pages 33-42, Newport Beach, CA, January 1992. Society for Computer Simulation.
  8. J. F. Ohi and B. R. Preiss. Parallel instance discrete-event simulation using a vector uniprocessor. In Proc. 1991 Winter Simulation Conf., pages 593-601, Phoenix, AZ, December 1991. Society for Computer Simulation.
  9. I. D. MacIntyre and B. R. Preiss. Multi-threaded pipelining in a RISC processor. In Proc. 1991 Canadian Conf. on Elec. and Comp. Eng., pages 7.3.1-7.3.6, Québec, P.Q., September 1991. Canadian Society for Electrical and Computer Engineering.
  10. B. R. Preiss, W. M. Loucks, I. D. MacIntyre, and J. A. Field. Null message cancellation in conservative distributed simulation. In Proc. 1991 Workshop on Parallel and Distributed Simulation, pages 33-38, Anaheim, CA, January 1991. Society for Computer Simulation.
  11. B. R. Preiss. Performance of discrete event simulation on a multiprocessor using optimistic and conservative synchronization. In Proc. 1990 Int. Conf. on Parallel Processing, pages 218-222, St. Charles, IL, August 1990. Penn. State University.
  12. B. R. Preiss and W. M. Loucks. The impact of lookahead on the performance of conservative distributed simulation. In Proc. 1990 European Multiconference--Simulation Methodologies, Languages and Architectures, pages 204-209, Nuremberg, FRG, June 1990. Society for Computer Simulation.
  13. W. M. Loucks and B. R. Preiss. The role of knowledge in distributed simulation. In Proc. SCS Multiconf. on Distributed Simulation, pages 9-16, San Diego, CA, January 1990. Society for Computer Simulation.
  14. B. R. Preiss. The Yaddes distributed discrete event simulation specification language and execution environments. In Proc. SCS Multiconf. on Distributed Simulation, pages 139-144, Tampa, FL, March 1989. Society for Computer Simulation.
  15. B. R. Preiss, W. M. Loucks, and V. C. Hamacher. A unified modeling methodology for performance evaluation of distributed discrete event simulation mechanisms. In Proc. 1988 Winter Simulation Conf., pages 315-324, San Diego, CA, December 1988. Society for Computer Simulation.
  16. B. R. Preiss and V. C. Hamacher. Semi-static dataflow. In Proc. 1988 Int. Conf. on Parallel Processing, volume 2, pages 127-134, St. Charles, IL, August 1988. Penn. State University.
  17. B. R. Preiss and V. C. Hamacher. A cache-based message passing scheme for a shared-bus multiprocessor. In Proc. 15th Int. Symp. on Computer Architecture, pages 358-364, Honolulu, HA, May 1988. Institute of Electrical and Electronics Engineers, Inc.
  18. B. R. Preiss and V. C. Hamacher. Data flow on a queue machine. In Proc. 12th Int. Symp. on Computer Architecture, pages 342-351, Boston, MA, August 1985. Institute of Electrical and Electronics Engineers, Inc.
  19. W. M. Loucks, V. C. Hamacher, B. R. Preiss, and L. Wong. Short-packet transfer performance in local area rings. In Proc. INFOCOM `84, pages 174-184, San Francisco, CA, April 1984. Institute of Electrical and Electronics Engineers, Inc.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Conference Papers (abstract refereed) Up: Publications Previous: Books

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Conference Papers (abstract refereed)

  1. J. M. Atlee, M. Godfrey, B. R. Preiss, and R. E. Seviora. Issues in joint undergraduate software engineering degree program design. In Proc. 1999 Canadian Conference on Computer Engineering Education, Vancouver, BC, June 1999. Computer Science Research, Education, and Applications Technology. (Proceedings to appear).
  2. D. Goswami, A. Singh, and B. R. Preiss. Architectural skeletons: The re-usable building-blocks for parallel applications. In Proc. 1999 International Conference on Parallel and Distributed Processing Techniques and Applications, volume 3, pages 1250-1256, Las Vegas, NV, June 1999. Computer Science Research, Education, and Applications Technology.
  3. S. K. Lindsay and B. R. Preiss. On the performance of a multi-threaded RISC architecture. In Proc. 1993 Canadian Conf. on Elec. and Comp. Eng., volume 1, pages 369-372, Vancouver, B.C., September 1993. Canadian Society for Electrical and Computer Engineering.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Submitted Conference Papers

  1. D. Goswami, A. Singh, and B. R. Preiss. Using object-oriented techniques for realizing parallel architectural skeletons. In Proc. ISCOPE '99, San Francisco, CA, December 1999.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Technical Reports

  1. B. R. Preiss. Amos third-party readiness initiative. Advanced technology report AT-GEN.0007, SOMA Networks, Inc., 2001.
  2. B. R. Preiss. Amos call processing model specification. Advanced technology report AT-GEN.0009, SOMA Networks, Inc., 2001.
  3. B. R. Preiss. Radically distributed call processing. Advanced technology proposal AT-PROP.0002, SOMA Networks, Inc., 2001.
  4. B. R. Preiss. SOMAport/PDA interface. Advanced technology proposal AT-PROP.0003, SOMA Networks, Inc., 2001.
  5. B. R. Preiss. SOMAport SIP client. Advanced technology proposal AT-PROP.0004, SOMA Networks, Inc., 2001.
  6. B. R. Preiss. Multi-release inter-operability static analysis tool. Advanced technology proposal AT-PROP.0005, SOMA Networks, Inc., 2001.
  7. B. R. Preiss. Core database/OSS interface architecture proposal. Engineering note EN-CORE.0006, SOMA Networks, Inc., 2001.
  8. B. R. Preiss. SOMAport terminal agent behavioral requirements. Engineering note EN-CP.0001, SOMA Networks, Inc., 2001.
  9. B. R. Preiss. Amos signaling events catalog. Engineering note EN-CP.0002, SOMA Networks, Inc., 2001.
  10. B. R. Preiss. Amos filter plug-ins catalog. Engineering note EN-CP.0003, SOMA Networks, Inc., 2001.
  11. B. R. Preiss. Amos call parameters catalog. Engineering note EN-CP.0004, SOMA Networks, Inc., 2001.
  12. B. R. Preiss. Amos IVR proposal. Engineering note EN-CP.0012, SOMA Networks, Inc., 2001.
  13. B. R. Preiss. Amos application host discovery. Engineering note EN-CP.0017, SOMA Networks, Inc., 2001.
  14. B. R. Preiss. Feature stacker proposal. Engineering note EN-CP.0019, SOMA Networks, Inc., 2001.
  15. B. R. Preiss. Changing filters in active calls. Engineering note EN-CP.0026, SOMA Networks, Inc., 2001.
  16. B. R. Preiss. Application host load shedding. Engineering note EN-CP.0027, SOMA Networks, Inc., 2001.
  17. B. R. Preiss. Amos fax/modem support. Engineering note EN-CP.0028, SOMA Networks, Inc., 2001.
  18. B. R. Preiss. Amos release inter-operability (Java). Engineering note EN-CP.0029, SOMA Networks, Inc., 2001.
  19. B. R. Preiss. Amos memory budget (application host). Engineering note EN-CP.0030, SOMA Networks, Inc., 2001.
  20. B. R. Preiss. Amos memory budget (SOMAport). Engineering note EN-CP.0031, SOMA Networks, Inc., 2001.
  21. B. R. Preiss. Amos vocoder support. Engineering note EN-CP.0032, SOMA Networks, Inc., 2001.
  22. B. R. Preiss. Out-of-band signaling for RTP support. Engineering note EN-CP.0034, SOMA Networks, Inc., 2001.
  23. B. R. Preiss and J. Pham. Amos services generic FSM. Engineering note EN-CP.0036, SOMA Networks, Inc., 2001.
  24. B. R. Preiss. Virtual time in the Amos FRE. Engineering note EN-CP.0039, SOMA Networks, Inc., 2001.
  25. B. R. Preiss. SOMAport FRE and IPAL filter paths. Engineering note EN-CP.0042, SOMA Networks, Inc., 2001.
  26. B. R. Preiss. Migrating to the new WSTEvent. Engineering note EN-CP.0045, SOMA Networks, Inc., 2001.
  27. B. R. Preiss. Executing call objects using a pool of worker threads. Engineering note EN-CP.0051, SOMA Networks, Inc., 2001.
  28. B. R. Preiss. Running multiple systems on an application host. Engineering note EN-CP.0053, SOMA Networks, Inc., 2001.
  29. B. R. Preiss. Specifying Amos filter paths using XML. Engineering note EN-CP.0055, SOMA Networks, Inc., 2001.
  30. B. R. Preiss and B. Gamsa. Amos diagnostics. Engineering note EN-CP.0059, SOMA Networks, Inc., 2001.
  31. B. R. Preiss. Amos entity names. Engineering note EN-CP.0061, SOMA Networks, Inc., 2001.
  32. B. R. Preiss. Amos filter path variants. Engineering note EN-CP.0062, SOMA Networks, Inc., 2001.
  33. B. R. Preiss. Amos web service architecture. Engineering note EN-CP.0064, SOMA Networks, Inc., 2001.
  34. B. R. Preiss. Amos end-to-end call flows -- release 1.0. Engineering note EN-CP.0065, SOMA Networks, Inc., 2001.
  35. B. R. Preiss. Amos configuration files. Engineering note EN-CP.0067, SOMA Networks, Inc., 2001.
  36. B. R. Preiss. Over-the-air service activation design document. Engineering note EN-CP.0070, SOMA Networks, Inc., 2001.
  37. B. R. Preiss. Specifying Amos regression test configurations. Engineering note EN-CP.0073, SOMA Networks, Inc., 2001.
  38. B. R. Preiss. Amos performance measurements and data generation. Engineering note EN-CP.0075, SOMA Networks, Inc., 2001.
  39. B. R. Preiss. Amos filter path catalog. Engineering note EN-CP.0077, SOMA Networks, Inc., 2001.
  40. B. R. Preiss. Decoupling classes for unit testing. Engineering note EN-CP.0081, SOMA Networks, Inc., 2001.
  41. B. R. Preiss. Voicemail lite. Engineering note EN-CP.0082, SOMA Networks, Inc., 2001.
  42. B. R. Preiss and D. Hsueh. Dummy services for unit testing. Engineering note EN-CP.0083, SOMA Networks, Inc., 2001.
  43. B. R. Preiss. POTS feature applet FSM specification. Engineering note EN-CP.0084, SOMA Networks, Inc., 2001.
  44. B. R. Preiss. POTS feature applet FSM diagram. Engineering note EN-CP.0085, SOMA Networks, Inc., 2001.
  45. B. R. Preiss. SOMAport terminal proxy FSM specification -- release 1.0. Engineering note EN-CP.0086, SOMA Networks, Inc., 2001.
  46. B. R. Preiss. SOMAport terminal, proxy, and FSM architecture -- release 1.0. Engineering note EN-CP.0087, SOMA Networks, Inc., 2001.
  47. B. R. Preiss. Supporting third-party services. Engineering note EN-CP.0097, SOMA Networks, Inc., 2001.
  48. B. R. Preiss. Amos event naming standard. Engineering note EN-CP.0099, SOMA Networks, Inc., 2001.
  49. B. R. Preiss. Filter path factory and filter path XML enhancements. Engineering note EN-CP.0100, SOMA Networks, Inc., 2001.
  50. B. R. Preiss. Pots feature applet specification -- version 2. Engineering note EN-CP.0101, SOMA Networks, Inc., 2001.
  51. B. R. Preiss. SOMAport terminal proxy specification -- version 2. Engineering note EN-CP.0105, SOMA Networks, Inc., 2001.
  52. B. R. Preiss. Sip terminal proxy specification -- version 2. Engineering note EN-CP.0114, SOMA Networks, Inc., 2001.
  53. B. R. Preiss. Amos end-to-end call flows -- version 2. Engineering note EN-CP.0114, SOMA Networks, Inc., 2001.
  54. B. R. Preiss. Amos system properties. Engineering note EN-CP.0114, SOMA Networks, Inc., 2001.
  55. B. R. Preiss. Option 4 -- moving the SOMAport terminal proxy FSM to the SOMAport. Engineering note EN-CP.0114, SOMA Networks, Inc., 2001.
  56. B. R. Preiss. Audio file prefetching. Engineering note EN-CP.0114, SOMA Networks, Inc., 2001.
  57. B. R. Preiss. Service activation flow. Engineering note EN-TR.0017, SOMA Networks, Inc., 2001.
  58. B. R. Preiss. IP port number assignments. Interface control document OPN-ICD.0010, SOMA Networks, Inc., 2001.
  59. B. R. P. M. Frazer and T. Savor. OpenNet software architecture. Architecture document OPN-SA.0003, SOMA Networks, Inc., 2001.
  60. B. R. Preiss and M. Thompson. Amos event logging service architecture. Architecture document OPN-SA.0004, SOMA Networks, Inc., 2001.
  61. B. R. Preiss and D. Wilk. Amos middleware and call processing architecture. Architecture document OPN-SA.0005, SOMA Networks, Inc., 2000.
  62. B. R. Preiss and M. Kamel. Amos feature configuration service architecture. Architecture document OPN-SA.0009, SOMA Networks, Inc., 2000.
  63. B. R. Preiss. Amos filter path architecture. Architecture document OPN-SA.0016, SOMA Networks, Inc., 2000.
  64. B. R. Preiss. Amos negotiation service architecture. Architecture document OPN-SA.0019, SOMA Networks, Inc., 2000.
  65. B. R. Preiss. Amos directory service architecture. Architecture document OPN-SA.0022, SOMA Networks, Inc., 2000.
  66. B. R. Preiss and M. Thompson. Amos agent registry service architecture. Architecture document OPN-SA.0023, SOMA Networks, Inc., 2000.
  67. B. R. Preiss. OpenNet core server architecture. Architecture document OPN-SA.0027, SOMA Networks, Inc., 2000.
  68. B. R. Preiss and M. Thompson. Amos boot service architecture. Architecture document OPN-SA.0028, SOMA Networks, Inc., 2001.
  69. B. R. Preiss. Amosphere security analysis. Architecture document OPN-SA.0071, SOMA Networks, Inc., 2001.
  70. B. R. Preiss. Amos SDK components. Architecture document OPN-SDK.0001, SOMA Networks, Inc., 2001.
  71. B. R. Preiss. Amos filter plug-in API. Architecture document OPN-SDK.0003, SOMA Networks, Inc., 2001.
  72. B. R. Preiss. Amos dsp filter plug-in API. Architecture document OPN-SDK.0004, SOMA Networks, Inc., 2001.
  73. B. R. Preiss. Amos feature applet API. Architecture document OPN-SDK.0006, SOMA Networks, Inc., 2001.
  74. B. R. Preiss. Amos services API. Architecture document OPN-SDK.0007, SOMA Networks, Inc., 2001.
  75. B. R. Preiss. Amos release 2 apis -- overview. Architecture document OPN-SDK.0008, SOMA Networks, Inc., 2002.
  76. B. R. Preiss and N. L. Shipp. Transputer system user guide--second edition. CCNG Technical Report E-240, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1995.
  77. B. R. Preiss, W. M. Loucks, and I. D. MacIntyre. Effects of the checkpoint interval on time and space in Time Warp. CCNG Technical Report E-230, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, June 1993.
  78. B. R. Preiss. Transputer system user guide. CCNG Technical Report E-212, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1992.
  79. B. R. Preiss and I. D. MacIntyre. YADDES--Yet Another Distributed Discrete Event Simulator: User manual. CCNG Technical Report E-197, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, 1990.
  80. B. R. Preiss and W. M. Loucks. Prediction and lookahead in distributed simulation. CCNG Technical Report E-191, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.
  81. B. R. Preiss. The Yaddes distributed discrete event simulation specification language and execution environments. CCNG Technical Report E-181, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.
  82. B. R. Preiss, W. M. Loucks, and V. C. Hamacher. A unified modeling methodology for performance evaluation of distributed discrete event simulation mechanisms. CCNG Technical Report E-180, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1989.
  83. B. R. Preiss and V. C. Hamacher. Semi-static dataflow. CCNG Technical Report E-170, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988.
  84. B. R. Preiss and V. C. Hamacher. A cache-based message passing scheme for a shared-bus multiprocessor. CCNG Technical Report E-169, Department of Electrical Engineering and Computer Communications Networks Group, University of Waterloo, 1988.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Theses Up: Publications Previous: Submitted Conference Papers

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Theses

  1. B. R. Preiss. Data Flow on a Queue Machine. PhD thesis, Department of Electrical Engineering, University of Toronto, 1987. 185 pp.
  2. B. R. Preiss. Design and Simulation of a Data-Flow Multiprocessor. Master's thesis, Department of Electrical Engineering, University of Toronto, 1984. 157 pp.
  3. B. R. Preiss. LARNS: A Powerful Model and Software Package for the Simulation of Local Area Ring Networks. Bachelor's thesis, Faculty of Applied Science and Engineering--Engineering Science Division (Electrical Engineering Option), University of Toronto, 1982. 122 pp.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Presentations, Seminars, Invited Talks

  1. The Joint CS and E&CE Proposal for a Software Engineering Undergraduate Degree Program. Seminar, PADS Research Group, University of Waterloo. June 1999.
  2. Issues in Joint Undergraduate Software Engineering Degree Program Design. Conference Presentation, Canadian Conference on Computer Engineering Education, Vancouver, BC. June 3, 1999.
  3. Design Patterns for the Data Structures and Algorithms Course. Conference Presentation, Proc. SIGCSE 1999 Technical Symposium, New Orleans, LA. March 25, 1999.
  4. The Parsimony Project: A Distributed Simulation Testbed in Java. Conference Presentation, Proc. 1999 International Conference On Web-Based Modelling & Simulation, San Francisco, CA. January 19, 1999.
  5. On the Design and Implementation of the Parsimony Checkpoint Package. Seminar, PADS Research Group, University of Waterloo. January 1999.
  6. C++ and Java: Comparing Apples with Oranges. Invited Talk, IEEE Canada, Kitchener-Waterloo Section, University of Waterloo. October 27, 1998.
  7. Parsimony: Parallel Simulator Once Named Yaddes. Seminar, PADS Research Group, University of Waterloo. May 1998.
  8. Java Inner Classes are Way Cool. Seminar, PADS Research Group, University of Waterloo. March 1998.
  9. Tricks of the Masters. Seminar, PADS Research Group, University of Waterloo. 1997.
  10. Java from the Perspective of a Long-Time C/C++ Hack. Seminar, PADS Research Group, University of Waterloo. 1997.
  11. Preaching/Teaching Objects from Day One. Conference Presentation, Canadian Conference on Computer Engineering Education, Waterloo, ON. May 9, 1997.
  12. On the Security of CGI Scripts. Seminar, PADS Research Group, University of Waterloo. 1996.
  13. An Algorithm for Speculative Parallel Execution of Rendezvous-Synchronized Simulation. Conference Presentation, 9th Workshop on Parallel and Distributed Simulation, Lake Placid, NY. June 17, 1995.
  14. Experiences in Parallel and Distributed Simulation. Invited Talk, IEEE Student Branch, University of Waterloo. March 14, 1995.
  15. Speculative Parallelism in Rendezvous Synchronized Simulation. Research Seminar, Royal Institute of Technology, Stockholm, Sweden. April 22, 1994.
  16. The Yaddes Project: Parallel Simulation Research at the University of Waterloo. Research Seminar, Royal Institute of Technology, Stockholm, Sweden. January 31, 1994.
  17. Selecting the Checkpoint Interval in Time Warp Simulation. Conference Presentation, 7th Workshop on Parallel and Distributed Simulation, San Diego, CA. May 16, 1993.
  18. On the Trade-off betweeen Time and Space in Optimistic Parallel Discrete-Event Simulation. Conference Presentation, 6th Workshop on Parallel and Distributed Simulation, Newport Beach, CA. January 22, 1922.
  19. Parallel Simulation of Continuous Systems using Time Warp. Research Seminar, VLSI Research Group Research Review, University of Waterloo. 1991.
  20. Null Message Cancellation in Conservative Distributed Simulation. Conference Presentation, 5th Workshop on Parallel and Distributed Simulation, Anaheim, CA. January 23, 1991.
  21. Parallel (Distributed) Discrete-Event Simulation. Seminar, Bell Northern Research (BNR), Ottawa, Canada. 1991.
  22. Yaddes Project Overview. Seminar, Cadence Computer Corp., Ottawa, Canada. 1990.
  23. Simulating Continuous Systems with Piecewise Linear Signals using Time Warp. Research Seminar, VLSI Research Group, University of Waterloo. 1990.
  24. Performance of Discrete Event Simulation on a Multiprocessor using Optimistic and Conservative Synchronization. Conference Presentation, 1990 Int. Conf. on Parallel Processing, Pheasant Run, IL. August 17, 1990.
  25. The Impact of Lookahead on the Performance of Conservative Distributed Simulation. Conference Presentation, 1990 European Simulation Multiconference, Nuremberg, FRG. June 10, 1990.
  26. The Yaddes Distributed Discrete Event Simulation Specification Language and Execution Environments. Conference Presentation, 3rd Workshop on Parallel and Distributed Simulation, Tampa, FL. March 28, 1989.
  27. Potential Harm of Introducing Advanced Computing Concepts in the Lower Years. Seminar, Engineering Education Research Centre (EERC), University of Waterloo. 1989.
  28. Progress in Distributed Simulation. Research Seminar, First Annual Information Technology Research Center (ITRC) Research Review, University of Toronto. 1989.
  29. A Unified Modeling Methodology for Performance Evaluation Of Distributed Discrete Event Simulation Mechanisms. Conference Presentation, 1988 Winter Simulation Conf., San Diego, CA. December 12, 1988.
  30. Semi-Static Dataflow. Conference Presentation, 1988 Int. Conf. on Parallel Processing, Pheasant Run, IL. August 15, 1988.
  31. A Distributed Discrete-Event Simulation Testbed. Research Seminar, VLSI Research Group Research Review, VLSI Research Group Research Review. 1988.
  32. Dataflow on a Queue Machine. Research Seminar, Department of Electrical Engineering, University of Waterloo. 1987.
  33. Dataflow on a Queue Machine. Ph.D. Thesis Oral Defense, Department of Electrical Engineering, University of Toronto. 1987.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Research Grants Up: Bruno Richard Preiss: Curriculum Previous: Theses

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Research Grants

  1. B. R. Preiss (PI). Network-Centric Distributed Discrete-Event Simulation. NSERC, Research Grant (individual, four year), $16113 (per annum), 1998-2002. (Status: Held).
  2. B. R. Preiss (PI). Parallel Co-simulation with Applications. NSERC, Research Grant (individual, four year) OGP 36635, $19656 (per annum), 1994-1998. (Status: Inactive).
  3. W. M. Loucks (PI), J. A. Field, and B. R. Preiss. Parallel and Distributed Simulation. ITRC, Research Grant, $8750, 1995-4-1-1995-6-30. (Status: Inactive).
  4. B. R. Preiss (PI), and R. Ayani. Svenska institutet (The Swedish Institute), Stipendium för utländsk forksares forskningsvistelse i Sverige (Stipend for foreign scientist's research visit in Sweden), 65800 SEK (CAN$10647), 1994-1-1-1994-6-30. (Status: Inactive).
  5. B. R. Preiss (PI). Axel Wenner-Gren Stiftelse, Stockholm, Sweden (Axel Wenner-Gren Foundation for Scientific Research), Grant in Aid of Research, 44400 SEK (CAN$7184), 1994-1-1-1994-6-30. (Status: Inactive).
  6. W. M. Loucks (PI), J. A. Field, and B. R. Preiss. Parallel and Distributed Simulation. ITRC, Research Grant, $70000, 1993-4-1-1995-3-31. (Status: Inactive).
  7. B. R. Preiss (PI), and W. M. Loucks. Message-based Multiprocessor System Expansion. NSERC, Equipment Grant EQP 107600, $33817, 1991. (Status: Inactive).
  8. B. R. Preiss (PI). Computer Architectures for Parallel Simulation. NSERC, Operating Grant OGP 36635, $15000 (per annum), 1991-1994. (Status: Inactive).
  9. W. M. Loucks (PI), J. A. Field, V. C. Hamacher, and B. R. Preiss. Hierarchical Distributed Processing Architecture. ITRC, Research Grant, $100000, 1991-4-1-1993-3-31. (Status: Inactive).
  10. W. M. Loucks (PI), and B. R. Preiss. Development of a Shared-Memory Multiprocessor Testbed. University of Waterloo, President's NSERC Fund, $10000, 1990. (Status: Inactive).
  11. J. W. Mark (PI), J. A. Field, G. B. Agnew, W. M. Loucks, and B. R. Preiss. Disk Upgrade. NSERC, Equipment Grant EQP 92156, $20844, 1990. (Status: Inactive).
  12. V. C. Hamacher (PI), J. A. Field, W. M. Loucks, and B. R. Preiss. Architecture and Language Support for Distributed Simulation. ITRC, Research Grant, $140000, 1989-1991. (Status: Inactive).
  13. V. C. Hamacher (PI), W. M. Loucks, and B. R. Preiss. Message Passing Architectures to Support Ambitious Processes. ITRC, Research Grant, $80000, 1988-1989. (Status: Inactive).
  14. B. R. Preiss (PI), and W. M. Loucks. Message-based Multiprocessor System. NSERC, Equipment Grant EQP 43585, $30334, 1989. (Status: Inactive).
  15. B. R. Preiss (PI). Dataflow Computer Architecture and Queue Machines. NSERC, Operating Grant OGP 36635, $11500 (per annum), 1988-1991. (Status: Inactive).
  16. B. R. Preiss (PI). University of Waterloo, Starter Grant, $5000, 1987. (Status: Inactive).

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Teaching Record Up: Bruno Richard Preiss: Curriculum Previous: PresentationsSeminars, Invited Talks

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Teaching Record

Table 1: Teaching Record
year | number | title | level | enrol. | Q10 | Q1-Q9 | % resp.
1999 W | E&CE 429 | Computer Structures | u/grad. | 98 | 88 | 86 | 51
1999 W | E&CE 301Q | Class Professor (class of 2001) | u/grad. |  |  |  |
1999 W | E&CE 250 | Algorithms and Data Structures | u/grad. | 94 | 85 | 85 | 71
1998 F | E&CE 652 | Parallel Programming | grad. | 13 credit
1998 F | E&CE 150 | Introduction to Computing | u/grad. | 129 | 83 | 80 | 84
1998 S | E&CE 202Q | Class Professor (class of 2001) | u/grad. |  |  |  |
1998 W | E&CE 429 | Computer Structures | u/grad. | 94 | 88 | 86 | 72
1998 W | E&CE 250 | Algorithms and Data Structures | u/grad. | 82 | 80 | 83 | 62
1997 F | E&CE 652 | Parallel Programming | grad. | 7 credit
1997 F | E&CE 250 | Algorithms and Data Structures | u/grad. | 91 | 92 | 90 | 54
1997 F | E&CE 201Q | Class Professor (class of 2001) | u/grad. |  |  |  |
1997 W | E&CE 429 | Computer Structures | u/grad. | 94 | 84 | 84 | 50
1997 W | E&CE 250 | Algorithms and Data Structures | u/grad. | 79 | 86 | 84 | 62
1997 W | E&CE 102Q | Class Professor (class of 2001) | u/grad. |  |  |  |
1996 F | E&CE 652 | Parallel Programming | grad. | 3 credit
1996 F | E&CE 250 | Algorithms and Data Structures | u/grad. | 80 | 85 | 85 | 79
1996 W | E&CE 429 | Computer Structures | u/grad. | 100 | 93 | 90 | 60
1996 W | E&CE 402Q | Class Professor (class of 1996) | u/grad. |  |  |  |
1996 W | E&CE 250 | Algorithms and Data Structures | u/grad. | 71 | 81 | 78 | 57
1995 F | E&CE 652 | Parallel Programming | grad. | 11 credit
1995 F | E&CE 250 | Algorithms and Data Structures | u/grad. | 78 | 93 | 88 | 59
1995 S | E&CE 401Q | Class Professor (class of 1996) | u/grad. |  |  |  |
1995 W | E&CE 429 | Computer Structures | u/grad. | 77 | 86 | 86 | 51
1995 W | E&CE 250 | Algorithms and Data Structures | u/grad. | 84 | 69 | 77 | 68
1994 F | E&CE 652 | Parallel Programming | grad. | 8 credit
1994 F | E&CE 302Q | Class Professor (class of 1996) | u/grad. |  |  |  |
1994 F | E&CE 250 | Algorithms and Data Structures | u/grad. | 79 | 81 | 79 | 46
1994 W | -- | Sabbatical | u/grad. | -- | -- | -- | --
1993 F | E&CE 652 | Parallel Programming | grad. | 11 credit/1 audit
1993 F | E&CE 621 | Computer Structures (substitute for 4 weeks) | grad. | 8 credit
1993 S | E&CE 251 | Programming Languages and Translators | u/grad. | 86 | 78 | 82 | 52
1993 W | E&CE 429 | Computer Structures | u/grad. | 68 | 79 | 82 | 62
1993 W | E&CE 324 | Microprocessor Systems and Interfacing | u/grad. | 73 | 89 | 88 | 77
1992 F | E&CE 652 | Parallel Programming | grad. | 5 credit/1 audit
1992 F | E&CE 250 | Algorithms and Data Structures | u/grad. | 80 | 70 | 76 | 41
1992 W | E&CE 429 | Computer Structures | u/grad. | 75 | 78 | 80 | 74
1992 W | E&CE 402Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1992 W | E&CE 324 | Microprocessor Systems and Interfacing | u/grad. | 77 | 86 | 80 | 68
1991 F | E&CE 652 | Parallel Programming | grad. | 9 credit/3 audit
1991 F | E&CE 251 | Programming Languages and Translators | u/grad. | 87 | 82 | 84 | 37
1991 S | E&CE 401Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1991 W | off | ITRC Teaching Relief | u/grad. | -- | -- | -- | --
1991 W | E&CE 429 | Computer Structures | u/grad. | 39 | 78 | 83 | 37
1990 F | E&CE 652 | Parallel Programming | grad. | 12 credit/3 audit
1990 F | E&CE 302Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1990 F | E&CE 251 | Programming Languages and Translators | u/grad. | 79 | 90 | 86 | 34
1990 W | ELE 429 | Computer Structures | u/grad. | 73 | 85 | 86 | 38
1990 W | ELE 324 | Microprocessor Systems and Interfacing | u/grad. | 93 | 87 | 85 | 50
1990 W | ELE 301Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1989 F | ELE 621 | Computer Organization | grad. | 7 credit/1 audit
1989 F | ELE 251 | Programming Languages and Translators | u/grad. | 82 | 78 | 80 | 60
1989 S | ELE 202Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1989 W | ELE 429 | Computer Structures | u/grad. | 75 | -- | -- | 0
1989 W | ELE 353 | Microprocessor Systems and Interfacing | u/grad. | 75 | 68 | 71 | 40
1988 F | ELE 429 | Computer Structures (Preparation Task) | u/grad. | -- | -- | -- | --
1988 F | ELE 251 | Programming Languages and Translators | u/grad. | 87 | 83 | 84 | 62
1988 F | ELE 201Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1988 W | ELE 725 | Dataflow Architectures and Machines | grad. | 4 credit/1 audit
1988 W | ELE 353 | Microprocessor Systems and Interfacing | u/grad. | 77 | 70 | 76 | 55
1988 W | ELE 102Q | Class Professor (class of 1992) | u/grad. |  |  |  |
1987 F | ELE 621 | Computer Organization | grad. | 8 credit

Table 1 gives the results of the Engineering Society Course Critiques for undergraduate courses. The column labelled Q10 gives the response to the question ``What is your overall appraisal of the quality of teaching in this course compared to others you've had at university?'' The column labelled Q1-Q9 is the average of the responses to nine other questions assessing various characteristics of the instructor, viz., organization and clarity of presentation, responses to questions, oral presentation ability, blackboard or overhead presentation, availability and approachability, level of explanations, motivational skills, attitude towards teaching, and the professor-class relationship.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Supervision of Students Up: Bruno Richard Preiss: Curriculum Previous: Research Grants

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Diplomas

  1. Secondary School Honours Graduation Diploma, 1978, Earl Haig Secondary School, North York, Ontario, Canada.
  2. Grade IX Piano (Honours), 1976, Royal Conservatory of Music, University of Toronto. Instructor: Roberta Dunn.
  3. Beigšanas Apliecıba (Graduation Certificate), 1977, Toronto Latviešu Tautas Augstskolas Sestdienas Ģimnāzija (High School of the Latvian National University of Toronto).
  4. Beigšanas Apliecıba (Graduation Certificate), 1973, Toronto Latviešu Biedrıbas Sestdienas Skola (Elementary School of the Toronto Latvian Association).

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Supervision of Students

  • Graduate Students
  • NSERC Undergraduate Summer Research Assistants/Co-op Students
  • UW Undergraduate Research Assistants
  • E&CE 499 Projects
  • Graduate Student Theses Supervised

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Sat Jan 26 21:21:33 EST 2002


Graduate Students

  1. Dhrubajyoti Goswami. Ph.D. candidate. September 1995-August 2001. Topic: Parallel Architectural Skeletons: Re-Usable Building Blocks for Parallel Applications. Status: Degree requirements completed. Supervisor: Ajit Singh and Bruno R. Preiss.
  2. Carey Ka Wing Wan. M.A.Sc. candidate (thesis). September 1997-December 1999. Topic: Towards a General Entity Framework for Distributed Interactive Simulation. Status: Degree conferred June 2000. Supervisor: Bruno R. Preiss.
  3. Mark J. Frazer. M.A.Sc. candidate (thesis). May 1996-April 1998. Topic: Towards Faster and Better Analytical Placements. Status: Degree conferred May 1998. Supervisor: Bruno R. Preiss and Andrew Kennings.
  4. Neil L. Shipp. M.A.Sc. candidate (thesis). May 1994-April 1996. Topic: Co-simulation of Combined Hardware Software Systems. Status: Degree conferred May 1996. Supervisor: Bruno R. Preiss. Co-supervised by Wayne M. Loucks during Prof. Preiss' sabbatical.
  5. Ka Kay Ho. M.A.Sc. candidate (thesis). September 1993-August 1995. Topic: Dynamic Rescheduling of Tasks in a Macro Dataflow Paradigm. Status: Degree conferred October 1995. Supervisor: Bruno R. Preiss. Co-supervised by Wayne M. Loucks during Prof. Preiss' sabbatical.
  6. James F. Ohi. M.A.Sc. candidate (project). September 1990-August 1993. Topic: On the Performance of Optimistic Parallel Discrete Event Simulation. Status: Degree conferred posthumously October 1993. Supervisor: Bruno R. Preiss.
  7. James F. Ohi. Ph.D. candidate. September 1991-August 1993. Topic: On the Performance of Optimistic Parallel Discrete Event Simulation. Status: Deceased. Supervisor: Bruno R. Preiss.
  8. Yipo Chi. Ph.D. candidate. May 1992-April 1993. Status: Degree conferred May 1995. Supervisor: Wayne M. Loucks. Co-supervised by Bruno R. Preiss during Prof. Loucks' sabbatical.
  9. Biswajit Nandy. Ph.D. candidate. May 1992-July 1993. Topic: Partitioning and Mapping for Parallel Discrete Event Simulation on Multiprocessors. Status: Degree conferred October 1993. Supervisor: Wayne M. Loucks. Co-supervised by Bruno R. Preiss during Prof. Loucks' sabbatical.
  10. Brad Kish. M.A.Sc. candidate (thesis). September 1991-August 1993. Topic: Hobbes: A Multi-Threaded Superscalar Processor. Status: Degree conferred October 1993. Supervisor: Bruno R. Preiss.
  11. Guoliang Tong. M.A.Sc. candidate (project). May 1992-April 1993. Topic: On Model Execution Overhead as an Estimate of Conservative Parallel Discrete Event Simulation Performance. Status: Degree conferred May 1993. Supervisor: Wayne M. Loucks. Co-supervised by Bruno R. Preiss during Prof. Loucks' sabbatical.
  12. Scott K. Lindsay. M.A.Sc. candidate (thesis). September 1990-August 1992. Topic: On the Performance of a Multi-Threaded RISC Architecture. Status: Degree conferred October 1992. Supervisor: Bruno R. Preiss.
  13. Naraig Manjikian. M.A.Sc. candidate (thesis). May 1992-1992. Topic: High Performance Parallel Logic Simulation on an Network of Workstations. Status: Degree conferred October 1992. Supervisor: Wayne M. Loucks. Co-supervised by Bruno R. Preiss during Prof. Loucks' sabbatical.
  14. Doug Milne. M.A.Sc. candidate (thesis). September 1989-April 1992. Topic: On the Design of a Process Resource Manager For Use in a Multiprocessor Processing Cluster. Status: Degree conferred May 1992. Supervisor: Bruno R. Preiss.
  15. Prabh Deep Singh. M.A.Sc. candidate (thesis). January 1990-January 1992. Topic: A Cache Coherence Protocol with Synchronization Mechanisms. Status: Degree conferred May 1992. Supervisor: Bruno R. Preiss.
  16. Alan Rooks. M.A.Sc. candidate (thesis). September 1989-August 1991. Topic: Routing and Cost Effectiveness of Directly-Connected Interconnection Networks. Status: Degree conferred October 1991. Supervisor: Bruno R. Preiss.
  17. James F. Ohi. M.A.Sc. candidate (thesis). September 1990-August 1991. Topic: On the Performance of Parallel Simulation. Status: Transferred to the Ph.D. programme. Supervisor: Bruno R. Preiss.
  18. Eyas Al-Hajery. M.A.Sc. candidate (thesis). January 1990-August 1990. Status: Withdrew from the programme. Supervisor: Bruno R. Preiss.
  19. Nicholas Swart. M.A.Sc. candidate (thesis). May 1990-August 1990. Status: Transferred to Prof. A. Nathan. Supervisor: Bruno R. Preiss.
  20. Barry T. W. Kwok. M.A.Sc. candidate (thesis). September 1988-January 1991. Topic: Distributed Gate Level Simulation Using Time Warp. Status: Degree conferred January 1991. Supervisor: Bruno R. Preiss.
  21. Ian D. MacIntyre. M.A.Sc. candidate (thesis). September 1987-October 1989. Topic: Cache Performance in a Horizontally Pipelined RISC Processor. Status: Degree conferred October 1989. Supervisor: Bruno R. Preiss.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: NSERC Undergraduate Summer Research Up: Supervision of Students Previous: Supervision of Students

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


NSERC Undergraduate Summer Research Assistants/Co-op Students

  1. Mark J. Frazer. NSERC Undergraduate Summer Research Assistant. September 1995-December 1995. Topic: Rendezvous-Synchronized Speculative Parallel Simulation. Supervisor: Bruno R. Preiss.
  2. Philip Chong. NSERC Undergraduate Summer Research Assistant. January 1995-April 1995. Topic: Hardware/Software Design. Supervisor: Wayne M. Loucks and Bruno R. Preiss.
  3. Neil L. Shipp. NSERC Undergraduate Summer Research Assistant. September 1993-December 1993. Topic: Parallel Co-Simulation using Multiple Workstations. Supervisor: Bruno R. Preiss.
  4. K. Whatmough. NSERC Undergraduate Summer Research Assistant. May 1990-August 1990. Topic: The WMMP Waterloo Multibus Multiprocessor. Supervisor: Bruno R. Preiss.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


UW Undergraduate Research Assistants

  1. Alim Somani. Undergraduate Research Assistant. 1999 Winter. Topic: Incremental Checkpointing for Time Warp Simulation. Supervisor: Bruno R. Preiss.
  2. Brigette Lau. Undergraduate Research Assistant. 1998 Spring. Topic: Distributed Simulation in Java. Supervisor: Bruno R. Preiss.
  3. David Chan. Undergraduate Research Assistant. 1998 Spring. Topic: Parsimony GUI in Java. Supervisor: Bruno R. Preiss.
  4. David Pritchard. Undergraduate Research Assistant. 1997 Fall. Topic: Java Virtual Machine Implementation. Supervisor: Bruno R. Preiss.
  5. Vineet Sinha. Undergraduate Research Assistant. 1997 Fall. Topic: Java Virtual Machine Implementation. Supervisor: Bruno R. Preiss.
  6. Mohammed A. Samji. Undergraduate Research Assistant. 1997 Winter. Topic: Literature Survey on Cosimulation. Supervisor: Bruno R. Preiss.
  7. Mark Gilbert. Undergraduate Research Assistant. 1996 Fall. Topic: Parallel Simulation using POSIX Threads. Supervisor: Bruno R. Preiss.
  8. David Tian. Undergraduate Research Assistant. 1996 Fall. Topic: Investigation of the Suitability of the Java Programming Language for Teaching Data Structures. Supervisor: Bruno R. Preiss.
  9. Brad Lynch. Undergraduate Research Assistant. 1996 Winter. Topic: Optimistic Parallel Simulation using Erlang. Supervisor: Bruno R. Preiss.
  10. Kenneth Tam. Undergraduate Research Assistant. 1996 Winter. Topic: Java Applet Frameworks for Algorithm Visualization. Supervisor: Bruno R. Preiss.
  11. Felix Wong. Undergraduate Research Assistant. 1995 Fall. Topic: Transputer Configuration Software. Supervisor: Bruno R. Preiss.
  12. George Huan. Undergraduate Research Assistant. 1995 Fall. Topic: Personal Communication System Simulation. Supervisor: Bruno R. Preiss.
  13. Brad Lynch. Undergraduate Research Assistant. 1995 Spring. Topic: Real-Time Parallel Simulation of Vibrating Membrane. Supervisor: Bruno R. Preiss.
  14. Mark J. Frazer. Undergraduate Research Assistant. 1995 Spring. Topic: Transputer Configuration Software. Supervisor: Bruno R. Preiss.
  15. Xiaohui James Chu. Undergraduate Research Assistant. 1995 Winter. Topic: Sound Synthesis by Real-Time Simulation. Supervisor: Bruno R. Preiss.
  16. Kenneth Chu. Undergraduate Research Assistant. 1994 Fall. Topic: Charm Port. Supervisor: Bruno R. Preiss.
  17. Neil Shipp. Undergraduate Research Assistant. 1993 Spring. Topic: Parallel Simulation on Workstations. Supervisor: Bruno R. Preiss.
  18. Stephen Williams. Undergraduate Research Assistant. 1992 Fall. Topic: Multi-workstation Logic Simulation. Supervisor: Bruno R. Preiss.
  19. Shamez Alibhai. Undergraduate Research Assistant. 1992 Spring. Topic: Waterloo Multibus-based Multiprocessor. Supervisor: Bruno R. Preiss.
  20. Neil Shipp. Undergraduate Research Assistant. 1991 Spring. Topic: Transputer Server Software. Supervisor: Bruno R. Preiss.
  21. Joseph Chen. Undergraduate Research Assistant. 1991 Winter. Topic: Waterloo Multibus Based Multi-Processor (WMMP) Power Supply Design. Supervisor: Bruno R. Preiss.
  22. Ashu Gupta. Undergraduate Research Assistant. 1990 Fall. Topic: Waterloo Multibus Based Multi-Processor (WMMP) Configuration Language. Supervisor: Bruno R. Preiss.
  23. Ming Ho. Undergraduate Research Assistant. 1989 Fall. Topic: 8086 Monitor Program. Supervisor: Bruno R. Preiss.
  24. Alex Leyn. Undergraduate Research Assistant. 1989 Spring. Topic: 8086 Monitor Program. Supervisor: Bruno R. Preiss.
  25. John Kim. Undergraduate Research Assistant. 1988 Winter. Topic: Multiprocessor Testbed Study. Supervisor: Bruno R. Preiss.
  26. Kevin Kornelson. Undergraduate Research Assistant. 1988 Winter. Topic: Multiprocessor Testbed Study. Supervisor: Bruno R. Preiss.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: E&CE 499 Projects Up: Supervision of Students Previous: NSERC Undergraduate Summer Research

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


E&CE 499 Projects

  1. R.-C. Shaw and A. Rogers. SY DE 462 Systems Design Workshop. 1999 Winter. Topic: CAROL: The Online Personal Information Manager.
  2. R. Caldart. E&CE 499 project. 1999 Winter. Topic: Incremental Checkpointing in Time Warp Simulation.
  3. M. Gilbert. E&CE 499 project. 1999 Winter. Topic: Digital Ink: Storage and Synchronization.
  4. T. Guttormson. E&CE 499 project. 1999 Winter. Topic: Shape Recognition and Conversion for a Pen-Based System.
  5. E. Thomas. E&CE 499 project. 1999 Winter. Topic: Digital Notepad Architecture and Pen-Based Interface.
  6. R. Leung. E&CE 499 project. 1999 Winter. Topic: Hardware Implementation of Garbage Collection in Java.
  7. R.-C. Shaw and A. Rogers. SY DE 461 Systems Design Workshop. 1998 Fall. Topic: CAROL: The Online Personal Information Manager.
  8. M. Gilbert. E&CE 499 project. 1998 Fall. Topic: Digital Ink: Storage and Synchronization.
  9. T. Guttormson. E&CE 499 project. 1998 Fall. Topic: Shape Recognition and Conversion for a Pen-Based System.
  10. E. Thomas. E&CE 499 project. 1998 Fall. Topic: Digital Notepad Architecture and Pen-Based Interface.
  11. R. Leung. E&CE 499 project. 1998 Spring. Topic: Hardware Implementation of Garbage Collection in Java.
  12. D. Zacher. E&CE 499 project. 1997 Winter. Topic: A Rule-Based Approach to Choral Arrangement.
  13. D. Villamere. E&CE 499 project. 1995 Winter. Topic: Messaging System: A Tool for Software Internationalization.
  14. P. Kim. E&CE 499 project. 1993 Winter. Topic: Graphical Simulation of Token Ring Protocol.
  15. B. Siim and J. Cherry. E&CE 499 project. 1992 Winter. Topic: Motorola 68030 VME Single-board Computer with MINIX Operating System.
  16. B. Siim and J. Cherry. E&CE 499 project. 1991 Spring. Topic: Motorola 68030 VME Single-board Computer with MINIX Operating System.
  17. N. Schellenberger. E&CE 499 project. 1992 Winter. Topic: Resolving Ambiguity in CLAY Grammars.
  18. N. Schellenberger. E&CE 499 project. 1991 Spring. Topic: Resolving Ambiguity in CLAY Grammars.
  19. T. R. M. Bramble and A. H. Reeves-Hall. E&CE 499 project. 1992 Winter. Topic: Multimedia and Object Oriented Design/Programming.
  20. T. R. M. Bramble and A. H. Reeves-Hall. E&CE 499 project. 1991 Spring. Topic: Multimedia and Object Oriented Design/Programming.
  21. A. Sim. E&CE 499 project. 1992 Winter. Topic: Transputer Board Project.
  22. A. Sim. E&CE 499 project. 1991 Spring. Topic: Transputer Board Project.
  23. P. Hoang. E&CE 499 project. 1991 Winter. Topic: The Design of a Message Coprocessor--WMC 2000.
  24. P. Hoang. E&CE 499 project. 1990 Spring. Topic: The Design of a Message Coprocessor--WMC 2000.
  25. C. Hang. E&CE 499 project. 1990 Winter. Topic: 8096 Monitor Program.
  26. W. Perrier. E&CE 499 project. 1990 Winter. Topic: IMST414/IMST800 Transputer Assembler Design Document and IMST414/IMST800 Transputer Assembler Input File Specification.
  27. W. Perrier. E&CE 499 project. 1989 Spring. Topic: IMST414/IMST800 Transputer Assembler Design Document and IMST414/IMST800 Transputer Assembler Input File Specification.
  28. A. Vadekar and D. O'Neill. E&CE 499 project. 1990 Winter. Topic: OC--An Occam Compiler.
  29. A. Vadekar and D. O'Neill. E&CE 499 project. 1989 Spring. Topic: OC--An Occam Compiler.
  30. S. Lindsay. E&CE 499 project. 1990 Winter. Topic: The WMMP Waterloo Multibus based Multi-Processor.
  31. S. Lindsay. E&CE 499 project. 1989 Spring. Topic: The WMMP Waterloo Multibus based Multi-Processor.
  32. J. F. Ohi. E&CE 499 project. 1989 Spring. Topic: The Design and Simulation of a General Purpose Array Processor.
  33. M. Wandel. E&CE 499 project. 1989 Spring. Topic: A Small Asynchronous Data Switch.
  34. S. F. Clarke and B. C. Crone. E&CE 499 project. 1989 Winter. Topic: Design Review of 8096 Disassembler Program (ARETHA III).
  35. S. G. Searles. E&CE 499 project. 1989 Winter. Topic: A Microprocessor-based Automotive Trip Computer.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Graduate Student Theses Supervised Up: Supervision of Students Previous: UW Undergraduate Research Assistants

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Graduate Student Theses Supervised

  1. D. Goswami. Parallel Architectural Skeletons: Re-Usable Building Blocks for Parallel Applications. PhD thesis, Department of Electrical and Computer Engineering, University of Waterloo, 2001. 147 pp.
  2. C. K. W. Wan. Towards a General Entity Framework for Distributed Interactive Simulation. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1999. 107 pp.
  3. M. J. Frazer. Towards Faster and Better Analytical Placements. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1998. 102 pp.
  4. D. Goswami. A design pattern based approach for developing parallel applications. Ph.D. Research Proposal, Department of Electrical and Computer Engineering, University of Waterloo, January 1997.
  5. N. L. Shipp. Co-simulation of Combined Hardware Software Systems. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1996. 103 pp.
  6. K. K. Ho. Dynamic Rescheduling of Tasks in a Macro Dataflow Paradigm. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1995. 99 pp.
  7. J. F. Ohi. On the Performance of Asynchronous Parallel Simulation. Master's project, Department of Electrical and Computer Engineering, University of Waterloo, 1993. 53 pp.
  8. B. J. Kish. Hobbes: A Multi-Threaded Superscalar Processor. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1993. 100 pp.
  9. S. K. Lindsay. On the Performance of a Multi-Threaded RISC Architecture. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1992. 96 pp.
  10. S. K. Lindsay. On the performance of a multi-threaded RISC architecture. CCNG Technical Report E-225, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, October 1992.
  11. D. Milne. On the Design of a Process Resource Manager for use in a Multiprocessor Processing Cluster. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1992. 129 pp.
  12. D. Milne. On the design of a process resource manager for use in a multiprocessor processing cluster. CCNG Technical Report T-218, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, June 1992.
  13. P. D. Singh. A Cache Coherence Protocol with Synchronization Mechanisms. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1992. 125 pp.
  14. J. F. Ohi. On the performance of asynchronous parallel simulation. Ph.D. Research Proposal, Department of Electrical and Computer Engineering, University of Waterloo, April 1992.
  15. A. Rooks. Routing and Cost Effectiveness of Directly-Connected Interconnection Networks. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1991. 111 pp.
  16. A. Rooks. Routing and cost effectiveness of directly-connected interconnection networks. CCNG Technical Report T-210, Department of Electrical and Computer Engineering and Computer Communications Networks Group, University of Waterloo, September 1991.
  17. B. T.-W. Kwok. Distributed Gate Level Simulation Using Time Warp. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1990. 111 pp.
  18. I. D. MacIntyre. A study of Horizontal Pipelining in a RISC Processor. Master's thesis, Department of Electrical and Computer Engineering, University of Waterloo, 1989. 163 pp.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Service to the University Up: Supervision of Students Previous: E&CE 499 Projects

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Service to the University of Waterloo

  1. Member (ex officio), Examinations and Promotions Committee, Faculty of Engineering, January 1999-August 1999.
  2. Member, First-Year Council, Faculty of Engineering, September 1998-June 1999.
  3. Member, CPH Multimedia Classroom Equipment and Furnishings Committee, Faculty of Engineering, May 1999-August 1999.
  4. Chair, E&CE 150/GEN E 121 Text Selection Committee, Faculty of Engineering, May 1999.
  5. Departmental Representative (elected), Engineering Faculty Council, Faculty of Engineering, 1988-1989, 1990-1991, 1992-1993, 1995-1996.
  6. Member, Faculty Graduate Studies Committee (Subcommittee A), Faculty of Engineering, 1995, 1996.
  7. Engineering Faculty Council Representative, Arts Faculty Council, Faculty of Engineering, 1995, 1996.
  8. Member, Computing Advisory Committee, Faculty of Mathematics, 1991-1993, July 1994-December 1994.
  9. Member, Advisory Committee to the Dean of Engineering on the Reappointment of the Chair of the Department of Electrical and Computer Engineering, Faculty of Engineering, 1995.
  10. Member, Web Design Working Group, Department of Electrical and Computer Engineering, March 1999-June 1999.
  11. Member, Joint CS/E&CE Steering Committee on the Proposed Software Engineering Degree Programme, Department of Electrical and Computer Engineering, March 1999-August 1999.
  12. Member, Software Technologist Hiring Committee, Department of Electrical and Computer Engineering, April 1999-June 1999.
  13. Member, Ad Hoc Committee to develop a Proposal for a Software Engineering Degree Programme, Department of Electrical and Computer Engineering, February 1998-March 1999.
  14. Member, Departmental Advisory Committee on Appointments (Computers, Digital Systems, and Software), Department of Electrical and Computer Engineering, May 1999-August 1999.
  15. Member, Departmental Advisory Committee on Appointments (Software Reliability Engineering), Department of Electrical and Computer Engineering, September 1998-August 1999.
  16. Computer Engineering Program Coordinator, Department of Electrical and Computer Engineering, January 1998-August 1999.
  17. Acting Coordinator, Theme Area B--Computing Systems, Hardware and Software, Department of Electrical and Computer Engineering, July 1998-December 1998.
  18. Adjunct Member, Fourth-Year Programme Restructuring Task Force, Department of Electrical and Computer Engineering, July 1998-October 1998.
  19. Member, Programme Management Task Force, Department of Electrical and Computer Engineering, December 1997-March 1998.
  20. Member, Future-Directions Task Committee, Department of Electrical and Computer Engineering, July 1997-September 1997.
  21. Member, Merit Review Committee (for Calendar Year 1996), Department of Electrical and Computer Engineering, January 1997.
  22. Member, Departmental Advisory Committee on Appointments--DACA C (Software Maintenance), Department of Electrical and Computer Engineering, March 1996-July 1996.
  23. Member, Promotions Committee, Department of Electrical and Computer Engineering, 1994-1995, 1995-1996, 1996-1997, 1997-1998.
  24. Secretary, Departmental Advisory Committee on Appointments (Bell Canada/NSERC Industrial Chair in Telecommunication Software Reliability Engineering), Department of Electrical and Computer Engineering, March 1995-November 1996.
  25. Member, Graduate Studies Committee, Department of Electrical and Computer Engineering, 1991-August 1993, September 1994-August 1996.
  26. Departmental Representative, Engineering Computing Committee, Faculty of Engineering, September 1995-August 1998.
  27. Member, Engineering Computing Joint Planning Committee, Faculty of Engineering, September 1995-August 1996.
  28. Computer Engineering Option Coordinator, Faculty of Engineering, September 1991-August 1998.
  29. Member, Ad Hoc Committee on Accreditation Matters, Department of Electrical and Computer Engineering, 1993.
  30. Member, Faculty Undergraduate Studies Committee, Faculty of Engineering, 1991, 1992, 1993.
  31. Secretary, Departmental Advisory Committee on Appointments (50s area), Department of Electrical and Computer Engineering, November 1991-December 1993.
  32. Coordinator, Teaching Assignments for 20s and 50s Courses, Department of Electrical and Computer Engineering, May 1992-December 1993, July 1994-December 1998.
  33. Member, Undergraduate Program Review Task Force, Department of Electrical and Computer Engineering, 1991.
  34. Member, Ad Hoc Committee on Departmental Evolution and Expansion, Department of Electrical and Computer Engineering, 1991.
  35. Member, Computer Task Force, Department of Electrical and Computer Engineering, 1987-1988.
  36. Member, Course Committee 1, Department of Electrical and Computer Engineering, 1987-1998.
  37. Member, Course Committee 3, Department of Electrical and Computer Engineering, 1987-1998.
  38. Member, Board of Directors, Institute for Computer Research (ICR), July 1991-June 1993, July 1994-June 1996.
  39. Member, Operations Committee, Institute for Computer Research (ICR), July 1991-June 1993, July 1994-June 1996.

Bruno Richard Preiss: Curriculum Vita
next up previous external
Next: Service to the Profession Up: Bruno Richard Preiss: Curriculum Previous: Graduate Student Theses Supervised

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Service to the Profession

  1. Member, Program Committee, [image] IEEE/SCS Simulation Symposium, 1997.
  2. Member, Program Committee, 1st World Congress on Systems Simulation (2nd Joint Conference of International Simulation Societies), 1997.
  3. Member, Collaborative Research and Development (CRD) Grant Selection Site Visit Committee, Natural Science and Engineering Research Council (NSERC) of Canada, 1995.
  4. Member, Program Committee, [image] IEEE/SCS Simulation Symposium, 1996.
  5. Member, Grant Selection Panel 306 (Electrical Engineering), Ontario Graduate Scholarship Programme, 1995.
  6. Co-Chair, Program Committee, [image] IEEE Workshop on Parallel and Distributed Simulation, 1996.
  7. Member, Program Committee, [image] IEEE/SCS Simulation Symposium, 1995.
  8. Member, Program Committee, [image] IEEE Workshop on Parallel and Distributed Simulation, 1994.
  9. Associate Editor, International Journal in Computer Simulation, ABLEX Publishing, January 1993-December 1994.
  10. Member, Program Committee, [image] IEEE/ACM International Symposium on Computer Architecture, 1991.
  11. Examiner, PEO Annual Examinations, 92-ELEC-A4, Digital Systems and Computers, Professional Engineers Ontario, 1994, 1995, 1996 (National).
  12. Examiner, APEO Annual Examinations, 86-ELEC-A5, Digital Systems I, Association of Professional Engineers of Ontario, 1991, 1992, 1993.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Scholarships and Awards

  1. 1982-1986, Open Doctoral Fellowship, University of Toronto.
  2. 1982, Postgraduate Scholarship, Natural Sciences and Engineering Research Council of Canada (NSERC).
  3. 1982, 1967 Science Scholarship, Natural Sciences and Engineering Research Council of Canada (NSERC).
  4. 1981, Undergraduate Summer Research Assistantship, Natural Sciences and Engineering Research Council of Canada (NSERC).
  5. 1981, Wallberg Undergraduate Scholarship, University of Toronto.
  6. 1979, Wallberg Undergraduate Scholarship, University of Toronto.
  7. 1978, George R. Mickle Admission Bursary, University of Toronto.
  8. 1978, Ontario Scholar, Province of Ontario.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Employment record

  1. August 1999-present, Software Engineer and Architect, Advanced Development, SOMA Networks, Inc., 312 Adelaide Street West, Suite 700, Toronto, Ontario, Canada, M5V 1R2.
  2. July 1993-August 2001 (On leave August 1999-August 2001), Associate Professor, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1.
  3. January 1994-June 1994, Gästforskare (Guest Researcher), Institutionen för Teleinformatik, Kungl. Tekniska Högskolan (Institute for Teleinformatics, Royal Institute of Technology), Electrum 204, 16440 Kista, Sweden.
  4. July 1987-June 1993, Assistant Professor, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1.
  5. January 1987-June 1987, Research Associate, Department of Electrical Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 1A4. Supervisor: Dr. Zvonko G. Vranesic.
  6. 1982-1986, Teaching assistant and laboratory demonstrator, University of Toronto, Department of Electrical Engineering, Toronto, Ontario, Canada, M5S 1A4. Courses: Computer Algorithms, Data Structures, and Languages (second year electrical engineering and engineering science); Computer Organization I (third year electrical engineering and engineering science); Computer Organization II (fourth year engineering science); and Digital Systems (second year electrical engineering).
  7. 1982 (summer), Member, Scientific Staff (summer), Bell Northern Research, Ottawa, Speech and Signal Processing. Supervisor: Dr. David Cuddy.
  8. 1981 (summer), Research Assistant, (NSERC Undergraduate Summer Research Assistantship), University of Toronto, Department of Electrical Engineering, Computer Group. Supervisor: Dr. V. Carl Hamacher.
  9. 1980 (summer), Research Assistant, University of Toronto, Department of Medical Biophysics, (at the Ontario Cancer Institute, Princess Margaret Hospital). Supervisor: Dr. R. M. Henkelman.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Membership in Professional Organizations and Societies

  1. Licenced Professional Engineer, Professional Engineers of Ontario (PEO).
  2. Member, Canadian Society for Electrical and Computer Engineering (CSECE).
  3. Member, Institute of Electrical and Electronics Engineers (IEEE).
  4. Member, Association for Computing Machinery (ACM).

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Publications

  • Refereed Full Journal Papers
  • Submitted Journal Papers
  • Refereed Technical Notes
  • Patents
  • Books
  • Conference Papers (full paper refereed)
  • Conference Papers (abstract refereed)
  • Submitted Conference Papers
  • Technical Reports
  • Theses

    Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
    Sat Jan 26 21:21:33 EST 2002


Refereed Full Journal Papers

  1. D. Goswami, A. Singh, and B. R. Preiss. From design patterns to parallel architecture skeletons. Journal of Parallel and Distributed Computing, June 2001. (To appear).
  2. B. R. Preiss, W. M. Loucks, and I. D. MacIntyre. Effects of the checkpoint interval on time and space in Time Warp. ACM Trans. on Modeling and Computer Simulation, 4(3):223-253, July 1994. (Accepted April 1994.).
  3. Y.-B. Lin and B. R. Preiss. Optimal memory management for Time Warp parallel simulation. ACM Trans. on Modeling and Computer Simulation, 1(4):283-307, October 1991. (Accepted May 1992. Published September 1992.).
  4. B. T. W. Kwok and B. R. Preiss. Simulating continuous systems with piecewise-linear signals using Time Warp. International Journal in Computer Simulation, 1(1):59-79, 1991.
  5. W. M. Loucks, V. C. Hamacher, B. R. Preiss, and L. Wong. Short packet transfer performance in local area ring networks. IEEE Trans. on Computers, C-34(11):1006-1014, November 1985.
  6. R. M. Henkelman and B. R. Preiss. A nonuniform detector aperture for CT. Journal of Computer Assisted Tomography, 5(3):401-408, June 1981.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Submitted Journal Papers

None as of January 26, 2002 .

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


Refereed Technical Notes

  1. R. M. Henkelman and B. R. Preiss. On the transport of charged particles through spongy materials. PhysMed. Biol., 26(6):1151-1156, June 1981.

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Sat Jan 26 21:21:33 EST 2002


 Resume of Bruno R. Preiss
next up previous
Up: Curriculum Vita

Résumé of Bruno R. Preiss

335 Calvington Place
Waterloo, ON N2T 1R1
(519) 725-2795
brpreiss@brpreiss.com
http://www.brpreiss.com

Employment

Software Engineer and Architect, Advanced Development
August 1999-present. SOMA Networks, Inc. Toronto, Ontario.

  • Principal software architect of the Amos distributed middleware and of the call processing (telephony) application.
  • Lead call processing team during expansion from 3 developers to over 20 developers.
  • Authored over 80 architecture documents, design documents, and engineering notes.
  • Directly influenced product definition, developed implementation plans, participated in sales presentations, and met with customer subject-matter experts.
  • Proposed, defined, and participated in advanced technology initiatives.
  • Co-inventor on five patent applications.

Associate Professor
July 1993-August 2001. (On leave August 1999-August 2001). Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario.

  • Conducted research in parallel and distributed computing. The primary focus was parallel and distributed simulation (PADS). This included the development of languages and run-time systems for PADS, algorithms and heuristics for synchronization and load balancing, the simulation of formal specifications, and web-based, distributed interactive simulation (DIS) using Java technology. Early research also addressed computer architectures, specifically the performance of multi-threaded RISC machines and their caches.
  • I was a leader in the use of the Internet and web-based technologies in support of teaching in the E&CE department at UW. I developed extensive websites with supporting materials for all the courses I taught.
  • Author of ``Data Structures and Algorithms with Object-Oriented Design Patterns in C++,'' John Wiley & Sons, 1999, 660 pp., and the accompanying website.
  • Author of ``Data Structures and Algorithms with Object-Oriented Design Patterns in Java,'' John Wiley & Sons, 2000, 635 pp., and the accompanying website.
  • Author or co-author of 7 journal papers, 22 conference papers, and 9 technical reports.
  • Principal Investigator or Associated Researcher on 15 research grants totaling in excess of $600,000.
  • Supervised or co-supervised 16 M.A.Sc. candidates and 4 Ph.D. candidates.
  • Taught the following courses: Introduction to Computer Programming (First-year Electrical Engineering (EE) and Computer Engineering (CE) programmes). Algorithms and Data Structures (Second-year EE and CE). Programming Languages and Translators (Second-year CE). Microprocessor Systems and Interfacing (Third-year CE). Computer Structures (Fourth-year CE). Computer Organization (Introductory grad course in EE). Dataflow Architectures and Machines (Advanced grad course in EE). Parallel Programming (Introductory grad course in EE).
  • Program committee member for 7 conferences including ``IEEE Workshop on Parallel and Distributed Simulation,'' ``IEEE Simulation Symposium'' and ``IEEE/ACM International Symposium on Computer Architecture.''
  • Examiner in ``Digital Systems and Computers'' for the annual examinations of Professional Engineers Ontario.
  • Member of the Board of Directors of the University of Waterloo Institute for Computer Research, 1991-1993 and 1994-1996.

Guest Researcher
January 1994-June 1994. Institute for Teleinformatics, Royal Institute of Technology, Kista, Sweden.

Assistant Professor
July 1987-June 1993. Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario.

Research Associate
January 1987-June 1987. Department of Electrical Engineering, University of Toronto, Toronto, Ontario.

Teaching Assistant/Laboratory Demonstrator
1982-1986 (part time). Department of Electrical Engineering, Toronto, University of Toronto, Toronto, Ontario.

Member, Scientific Staff
1982 (summer). Speech and Signal Processing Department, Bell Northern Research, Ottawa, Ontario.

Research Assistant
1981 (summer). Department of Electrical Engineering, Computer Group, University of Toronto, Toronto, Ontario.

Research Assistant
1980 (summer). Department of Medical Biophysics, University of Toronto, Toronto, Ontario.

Education

Ph.D.
1987, University of Toronto, Department of Electrical Engineering. Dissertation title: ``Dataflow on a Queue Machine.''

M.A.Sc.
1984, University of Toronto, Department of Electrical Engineering. Dissertation title: ``Design and Simulation of a Data-Flow Multiprocessor System.''

B.A.Sc.
1982, University of Toronto, Engineering Science Division, (Electrical Engineering Option).

P.Eng.
Licensed Professional Engineer in the Province of Ontario.

Computing Experience and Skills

Parallel/Distributed Systems
Research and teaching experience in parallel, concurrent, and distributed applications development.
Programming Languages and Tools
Expert C programmer. Expert in object-oriented programming using C++ and Java. Proficient in scripting languages such as Perl, Awk, Sed, Csh. Familiar with Pascal, Fortran. Experience with compiler development tools including Yacc and Lex.
Operating Systems
Expert knowledge of Unix. Experience with Unix system administration. Proficient with DOS/Windows.
Internet
Experience with HTML including CGI scripts and forms. Familiar with Java applet development and JSP. Experience with XML parsers and generators.
Desktop applications
Expert with LaTeX. Proficient with Framemaker, Word, Excel, and Powerpoint. Experience with Access and Project.

Resume of Bruno R. Preiss
next up previous
Up: Curriculum Vita

Bruno Copyright © 2002 by Bruno R. Preiss, P.Eng. All rights reserved.
Fri Jan 25 15:12:25 EST 2002


                        Resume of Bruno R. Preiss

                         335 Calvington Place
                         Waterloo, ON N2T 1R1
                            (519) 725-2795
                        brpreiss@brpreiss.com
                       http://www.brpreiss.com

                               Employment

Software Engineer and Architect, Advanced Development
       August 1999-present. SOMA Networks, Inc. Toronto, Ontario.

       + Principal   software   architect   of  the  Amos  distributed
         middleware   and   of   the   call   processing   (telephony)
         application.
       + Lead  call processing team during expansion from 3 developers
         to over 20 developers.
       + Authored  over  80  architecture documents, design documents,
         and engineering notes.
       + Directly    influenced    product    definition,    developed
         implementation  plans,  participated  in sales presentations,
         and met with customer subject-matter experts.
       + Proposed,  defined,  and  participated in advanced technology
         initiatives.
       + Co-inventor on five patent applications.

Associate Professor
       July  1993-August  2001.  (On  leave  August 1999-August 2001).
       Department  of  Electrical and Computer Engineering, University
       of Waterloo, Waterloo, Ontario.

       + Conducted research in parallel and distributed computing. The
         primary focus was parallel and distributed simulation (PADS).
         This  included  the  development  of  languages  and run-time
         systems    for    PADS,   algorithms   and   heuristics   for
         synchronization  and load balancing, the simulation of formal
         specifications,   and   web-based,   distributed  interactive
         simulation  (DIS)  using Java technology. Early research also
         addressed    computer    architectures,    specifically   the
         performance of multi-threaded RISC machines and their caches.
       + I  was  a  leader  in  the  use of the Internet and web-based
         technologies in support of teaching in the E&CE department at
         UW.  I developed extensive websites with supporting materials
         for all the courses I taught.
       + Author    of    ``Data   Structures   and   Algorithms   with
         Object-Oriented  Design Patterns in C++,'' John Wiley & Sons,
         1999, 660 pp., and the accompanying website.
       + Author    of    ``Data   Structures   and   Algorithms   with
         Object-Oriented Design Patterns in Java,'' John Wiley & Sons,
         2000, 635 pp., and the accompanying website.
       + Author  or  co-author  of  7  journal  papers,  22 conference
         papers, and 9 technical reports.
       + Principal   Investigator   or  Associated  Researcher  on  15
         research grants totaling in excess of $600,000.
       + Supervised or co-supervised 16 M.A.Sc. candidates and 4 Ph.D.
         candidates.
       + Taught   the  following  courses:  Introduction  to  Computer
         Programming   (First-year  Electrical  Engineering  (EE)  and
         Computer  Engineering  (CE)  programmes). Algorithms and Data
         Structures (Second-year EE and CE). Programming Languages and
         Translators  (Second-year  CE).  Microprocessor  Systems  and
         Interfacing (Third-year CE). Computer Structures (Fourth-year
         CE).  Computer Organization (Introductory grad course in EE).
         Dataflow  Architectures and Machines (Advanced grad course in
         EE). Parallel Programming (Introductory grad course in EE).
       + Program  committee  member for 7 conferences including ``IEEE
         Workshop  on  Parallel  and  Distributed Simulation,'' ``IEEE
         Simulation Symposium'' and ``IEEE/ACM International Symposium
         on Computer Architecture.''
       + Examiner  in ``Digital Systems and Computers'' for the annual
         examinations of Professional Engineers Ontario.
       + Member  of  the  Board  of  Directors  of  the  University of
         Waterloo  Institute  for  Computer  Research,  1991-1993  and
         1994-1996.

Guest Researcher
       January  1994-June  1994.  Institute for Teleinformatics, Royal
       Institute of Technology, Kista, Sweden.

Assistant Professor
       July  1987-June  1993.  Department  of  Electrical and Computer
       Engineering, University of Waterloo, Waterloo, Ontario.

Research Associate
       January  1987-June  1987. Department of Electrical Engineering,
       University of Toronto, Toronto, Ontario.

Teaching Assistant/Laboratory Demonstrator
       1982-1986  (part  time).  Department of Electrical Engineering,
       Toronto, University of Toronto, Toronto, Ontario.

Member, Scientific Staff
       1982  (summer).  Speech  and Signal Processing Department, Bell
       Northern Research, Ottawa, Ontario.

Research Assistant
       1981  (summer).  Department of Electrical Engineering, Computer
       Group, University of Toronto, Toronto, Ontario.

Research Assistant
       1980  (summer). Department of Medical Biophysics, University of
       Toronto, Toronto, Ontario.

                                Education

Ph.D.
       1987,   University   of   Toronto,   Department  of  Electrical
       Engineering.   Dissertation   title:   ``Dataflow  on  a  Queue
       Machine.''

M.A.Sc.
       1984,   University   of   Toronto,   Department  of  Electrical
       Engineering.  Dissertation  title: ``Design and Simulation of a
       Data-Flow Multiprocessor System.''

B.A.Sc.
       1982,  University  of  Toronto,  Engineering  Science Division,
       (Electrical Engineering Option).

P.Eng.
       Licensed Professional Engineer in the Province of Ontario.

                     Computing Experience and Skills

Parallel/Distributed Systems
       Research  and  teaching experience in parallel, concurrent, and
       distributed applications development.

Programming Languages and Tools
       Expert  C  programmer.  Expert  in  object-oriented programming
       using  C++  and Java. Proficient in scripting languages such as
       Perl,  Awk, Sed, Csh. Familiar with Pascal, Fortran. Experience
       with compiler development tools including Yacc and Lex.

Operating Systems
       Expert   knowledge   of   Unix.  Experience  with  Unix  system
       administration. Proficient with DOS/Windows.

Internet
       Experience  with HTML including CGI scripts and forms. Familiar
       with  Java  applet  development  and  JSP.  Experience with XML
       parsers and generators.

Desktop applications
       Expert with LaTeX. Proficient with Framemaker, Word, Excel, and
       Powerpoint. Experience with Access and Project.