1 .\" $Id: val.doc,v 1.5 1994/06/24 10:02:39 ceriel Exp $
92 \fBPascal Validation Suite Report\fR
93 .CH "Pascal processor identification"
94 The ACK-Pascal compiler produces code for an EM machine
96 It is up to the implementor of the EM machine whether errors like
97 integer overflow, undefined operand and range bound error are recognized or not.
98 Therefore it depends on the EM machine implementation whether these errors
99 are recognized in Pascal programs or not.
100 The validation suite results of all known implementations are given.
102 There does not (yet) exist a hardware EM machine.
103 Therefore, EM programs must be interpreted, or translated into
104 instructions for a target machine.
105 The following implementations currently exist:
108 an interpreter running on a PDP-11 (using UNIX).
109 The normal mode of operation for this interpreter is to check
110 for undefined integers, overflow, range errors etc.
113 a translator into PDP-11 instructions (using UNIX).
114 Less checks are performed than in the interpreter, because the translator
115 is intended to speed up the execution of well-debugged programs.
117 .CH "Test Conditions"
122 Validation Suite version: 3.0
124 The final test run is made with a slightly
125 modified validation suite.
126 .SH "Erroneous programs"
127 Some test did not conform to the standard proposal of February 1979.
128 It is this version of the standard proposal that is used
129 by the authors of the validation suite.
132 The semicolon between high and integer on line 17 is replaced
136 The div operator on line 14 replaced by mod.
137 .CH "Conformance tests"
138 Number of tests passed = 150
140 Number of tests failed = 6
141 .SH "Details of failed tests"
144 Character sequences starting with the 8 characters 'procedur'
146 erroneously classified as the word-symbols 'procedure' and 'function'.
149 Identifiers identical in the first eight characters, but
150 differing in ninth or higher numbered characters are treated as
154 ACK-Pascal requires all formal program parameters to be
155 declared with type \fIfile\fP.
158 Gives run-time error eof seen at call to eoln.
159 A have a hunch that this is a error in the suit.
162 Redefining the names of some standard procedures leads to incorrect
163 behaviour of the runtime system.
164 In this case it crashes without a sensible error message.
167 This test can not be translated by our compiler because two
168 non-identical variables are used in the same block with the same first eight
170 The test passed after replacement of one of those names.
173 Number of deviations correctly detected = 120
175 Number of tests not detecting deviations = 20
176 .SH "Details of deviations"
177 The following tests are compiled without a proper error
178 indication although they do
179 not conform to the standard.
182 ACK-Pascal allows labels in the range 0..32767.
183 A warning is produced when testing for deviations from the
187 A missing space between a number and a word symbol is not
195 Undetected scope error. The scope of an identifier should start at the
196 beginning of the block in which it is declared.
197 In the ACK-Pascal compiler the scope starts just after the declaration,
201 The values of fields from one variant are accessible from
203 The correlation is exact.
206 The passing as a variable parameter of the selector of a
207 variant part is not detected.
208 A runtime error is produced because the variant selector is not
216 The ACK-Pascal compiler does not restrict the places from where
217 a jump to a label by means of a goto-statement is allowed.
223 There are no errors produced for assignments to a variable
224 in use as control-variable of a for-statement.
227 Use of a controlled variable after leaving the loop without
228 intervening initialization is not detected.
231 The results depend on the EM implementation.
233 Number of errors correctly detected =
240 Number of errors not detected =
247 Number of errors incorrectly detected =
254 .SH "Details of errors not detected"
255 The following test fails because the ACK-Pascal compiler only
256 generates a warning that does not prevent to run the tests.
259 A warning is produced if there is no assignment to a function-identifier.
261 With this test the ACK-Pascal compiler issues an error message for a legal
262 construct not directly related to the error to be detected.
265 Program does not compile.
266 Buffer variable of text file is not allowed as variable
269 The following errors are not detected at all.
273 The use of an undefined integer is not caught as an error.
279 The notion of 'current variant' is not implemented, not even if a tagfield
288 Subrange bounds are not checked.
293 If the base-type of a set is a subrange, then the set elements are not checked
294 against the bounds of the subrange.
295 Only the host-type of this subrange-type is relevant for ACK-Pascal.
299 Nil pointers are not detected.
303 Undefined pointers are not detected.
306 Changing the file position while the window is in use as actual variable
307 parameter or as an element of the record variable list of a with-statement
311 An undefined function result is not detected,
312 because it is never used in an expression.
316 Disposing a variable while it is in use as actual variable parameter or
317 as an element of the record variable list of a with-statement is not detected.
322 It is not detected that a record variable, created with the variant form
323 of new, is used as an operand in an expression or as the variable in an
324 assignment or as an actual value parameter.
327 Use of a variable that is not reinitialized after a dispose is
334 There are no range checks for pred, succ and chr.
337 ACK-Pascal considers a rewrite of a file as a defining
345 Division by 0 or integer overflow is not detected.
348 The use of the some control variable in two nested for
349 statements in not detected.
352 Access of a control variable after leaving the loop results in
353 the final-value, although an error should be produced.
356 The program stops with a file not open error.
357 The rewrite before the write is missing in the program.
361 Illegal FracDigits values are not detected.
362 .CH "Implementation dependence"
363 Number of tests run = 14
365 Number of tests incorrectly handled = 0
366 .SH "Details of implementation dependence"
369 Alternate comment delimiters are implemented
372 The equivalent symbols @ for ^, (. for [ and .) for ] are not
379 Only elements with non-negative ordinal value are allowed in sets.
382 Standard procedures and functions are not allowed as parameters.
385 Details of the machine characteristics regarding real numbers:
398 epsneg = 1.387779e-17
406 All operands of boolean expressions are evaluated.
410 The expression in an assignment statement is evaluated
411 before the variable selection if this involves pointer
412 dereferencing or array indexing.
415 Actual parameters are evaluated in reverse order.
418 The default width for integer, Boolean and real are 6, 5 and 13.
421 The number of digits written in an exponent is 2.
424 The representations of true and false are (~true) and (false).
425 The parenthesis serve to indicate width.
427 .CH "Quality measurement"
428 Number of tests run = 60
430 Number of tests handled incorrectly = 1
431 .SH "Results of tests"
432 Several test perform operations on reals on indicate the error
433 introduced by these operations.
434 For each of these tests the following two quality measures are extracted:
437 maxRE:~~maximum relative error
439 rmsRE:~~root-mean-square relative error
445 25 thousand Whetstone instructions per second.
447 169 thousand Whetstone instructions per second.
450 The value of (TRUEACC-ACC)*2^56/100000 is 1.4 .
451 This is well within the bounds specified in [3].
460 The number of procedure calls calculated in this test exceeds
461 the maximum integer value.
462 The program stops indicating overflow.
465 The number of significant characters for identifiers is 8.
468 There is no maximum to the line length.
471 The error message "too many digits" is given for numbers larger
477 Normal values are allowed for real constants and variables.
480 A reasonably large number of strings is allowed.
483 No warning is given for possibly unclosed comments.
490 Large lists of declarations are possible in each block.
493 An 'array[integer] of' is not allowed.
497 Large values are allowed for arrays and indices.
500 Large amounts of case-constant values are allowed in variants.
503 Large amounts of record sections can appear in the fixed part of
507 Large amounts of variants are allowed in a record.
509 Size and speed of Warshall's algorithm depend on the
510 implementation of EM:
525 Deep nesting of array indices is allowed.
529 Arrays can have at least 8 dimensions.
532 Deep static nesting of procedure is allowed.
535 Large amounts of formal parameters are allowed.
538 Dispose is fully implemented.
541 Test sqrt(x): no errors.
542 The error is within acceptable bounds.
550 Test arctan(x): may cause underflow or overflow errors.
551 The error is within acceptable bounds.
560 Test exp(x): may cause underflow or overflow errors.
561 The error is not within acceptable bounds.
569 Test sin(x): may cause underflow errors.
570 The error is not within acceptable bounds.
577 Test cos(x): may cause underflow errors.
578 The error is not within acceptable bounds.
587 The error is not within acceptable bounds.
597 Complex nested expressions are allowed.
601 The error is within acceptable bounds.
609 Operations of reals in the integer range are exact.
619 Static deep nesting of function calls,
620 compound statements, if statements, case statements, repeat
621 loops, while loops, for loops and with statements is possible.
624 Large amounts of statements are allowed in a compound
628 The compiler requires case constants to be compatible with
633 Large case statements are possible.
636 Recursive IO on the same file is well-behaved.
639 The reading of real values from a text file is done with
651 Read, readln, write and writeln may have large amounts of
655 The loss of precision for reals written on a text file and read
664 File IO buffers without trailing marker are correctly flushed.
667 Reals are written with sufficient accuracy.
674 .CH "Level 1 conformance tests"
675 Number of test passed = 4
677 Number of tests failed = 1
678 .SH "Details of failed tests"
681 An expression indicated by parenthesis whose
682 value is a conformant array is not allowed.
684 .CH "Level 1 deviance tests"
685 Number of deviations correctly detected = 4
687 Number of tests not detecting deviations = 0
689 .CH "Level 1 error handling"
690 The results depend on the EM implementation.
692 Number of errors correctly detected =
699 Number of errors not detected =
706 .SH "Details of errors not detected"
710 Subrange bounds are not checked.
712 .CH "Level 1 quality measurement"
713 Number of tests run = 1
714 .SH "Results of test"
717 Large conformant arrays are allowed.
720 Number of tests run = 3
721 .SH Details of test failed
724 The alternative relational operators are not allowed.
727 The alternative symbols for colon, semicolon and assignment are
731 The otherwise selector in case statements is not allowed.
736 A.S.Tanenbaum, E.G.Keizer, J.W.Stevenson, Hans van Staveren,
737 "Description of a machine architecture for use with block structured
739 Informatica rapport IR-81.
742 ISO standard proposal ISO/TC97/SC5-N462, dated February 1979.
743 The same proposal, in slightly modified form, can be found in:
744 A.M.Addyman e.a., "A draft description of Pascal",
745 Software, practice and experience, May 1979.
746 An improved version, received March 1980,
747 is followed as much as possible for the
751 B. A. Wichman and J du Croz,
752 A program to calculate the GAMM measure, Computer Journal,
public / Ack-5.5.git / blob