
# SID   @(#) coding.txt 3.8 24/08/18 13:40:55

# Description about Program Code and Documentation
# It is not shown with +help but can be retrieved with: $0 --help=ProgramCode
# Note that  $0  is used herein for the tool. It will be replaced by the
# proper tool name when called by  $0  itsef.

HACKER's INFO

    Terminology

        Throughout the code and its documentation following terms are used:

          * <>  - angle  brackets (chevron)
          * {}  - curly  brackets (brace, accolades, llave)
          * ()  - round  brackets (paranthese, paranthèse, paréntesis)
          * []  - square brackets (bracket, crochet, corchete)

        This description does not use text decorations for  Perl keywords, as
        they are most likely identified by the context.

    Program Code

        First of all: the main goal is to have a tool to be simple for users.
        It's not designed to be academic code or simple for programmers.

        Also testing for various flaws in other tools and protocols could not
        be done in  a standardised generic way, using well designed software,
        but often needs highly adapted code for each individual check, and --
        more worse -- sometimes variants of the same code.
        Please keep this in mind, before trying to unitise the code.

        Other people may have other programming styles ...

        Note:  following descriptions mainly uses the term "sub" (the perlish
        term) when talking about functions, procedures, and/or methods.

      Syntax style

        Identation is 4 spaces. TABs would be the better solution, IMHO.
        Unfortunately some repositories have issues with TABs,  so spaces are
        used only. Sick.

        Additional spacing is used to format the code for better human reada-
        bility. There is no strict rule about this, it's just done as needed.

        Empty lines (empty, without any space!) are used to group code blocks
        logically. However, there is no strict rule about this too.

        K&R-style curly brackets for subs and conditions are used.
        K&R-style round brackets for subs and conditions are used (means that
        definitions of subs, and calls of subs  do not use  spaces before the
        opening bracket, while conditions use spaces).

        Calls of subs are written in  K&R-style using round brackets, and the
        perlisch way without round brackets. This may be unitised in future.
        Main exception is  print, which is often used without brackets.

        Short subs and conditions may be written in just one line.
        Note: there is no need for each command in its one line, as debugging
        on code level is rarely done.

        subs are defined with number and type of parameters to have a minimal
        syntax check at compile time.
        This will be changed in future.

        For quoting see below.

      Brackets

        Angle, curly and square brackets are mainly unambiguous in Perl.

        Round brackets are unambiguous if used for grouping.  Parameters  for
        function calls are also a group. Hence they're most likely used there
        even Perl does not require them.  This avoids unambiguous, unexpected
        use of parameters, for example when a function's return value is used
        as parameter for another function (in-line code).

        Examples of ambiguous code to be avoided:

          * 'print 4, 2, rand 1, 0;'    # 1 is paramter for rand not print
          * '&myfunc;'                  # call uses @_
          * 'myfunc;'                   # maybe a function call ...
          * '$f="foo"; $bar->foo;'      # do all humans see the function?
          * '$f="foo"; $bar->$f();'     # round brackets mandatory

        Round brackets are sometimes omitted for:

          * LHS condition
          * built-in functions, like: keys, print, sort, scalar

        However, if multiple functions are nested,  round brackets are use to
        avoid unexpected behaviour. For example:
            reverse(sort(keys %hash));
        instead of perlisch:
            reverse sort keys %hash;

      Code style

        In general the code favours language constructs that are easy to read
        by humans, even they are unusual for other programming languages (for
        example postfix conditions, see below).

        Global variables are used, see X&Variables& for details below.

        Variables are declared at beginning of subs. I.g. we do not use local
        or my declarations in blocks (there may be some exceptions).

        The code tries to avoid if-else constructs as much as possible. If an
        else condition is used, it is written in one line:   } else {  .
        elsif is used in "borrowed" code only ;-)

        if-constructs are used to deal with exceptions of the  normal control
        flow. The flow will continue behind the loop  or exit the loop or sub
        using: break, continue, last, next, return.

        As described before, "early breaks" (next, last in loops; return from
        function) are prefered, rather than complicated or nested conditions.
        There are also goto statements in parsers,  but return statements are
        prefered.

        Most code is seqential instead of using functions, except the code is
        used multiple times. This is currenlty changed ...
        It is not intended to have OO-code,  even Perl's  OO capabilities are
        used when rational.

      Code quality

        The code is regulary analysed using "perlcritic" -a front-end command
        line tool to the Perl::Critic module. As decribed above, this code is
        not supposed to be academically correct. Nevertheless, we follow most
        recommendations given by perlcritic to make the most of it.
        For details please see:

          *  Annotations at end of $0
          *  t/.perlcriticrc
          *  usr/critic.sh .

        All tests can be done with make, see:

          *  make help.test.critic

      Quoting

        Using single quotes  '  and/or double quotes  "  is the nature of the
        processed data, the generated data or the program code.  These quotes
        are written literally in data, whereas  Perl provides various methods
        to handles different types of quotes. Coding gets complicated if both
        quote types are part of the data, in particular when printing data.

        The  code  mainly uses  'text enclosed in single quotes'  for program
        internal strings such as hash keys. It uses  "double quoted text" for
        texts being printed. However, exceptions if obviously necessary ;-)
        Strings used for  RegEx are always enclosed in single quotes.  Reason
        is mainly to make searching texts a bit easier.

        Perl's operators  q() and  qq()  are used  only to avoid  escaping of
        character, because all types of quotes can smoothly mixed there.
        In general  "double quoted text" is used for interpolated strings. In
        some cases, it is necessary to use Perl's  sprintf()  function.
        For details, please see:

            perldoc -f q
            perldoc -f qq
            perldoc -f sprintf
            perldoc perlop # following sections:
                Comma Operator
                Quote and Quote-Like Operators
                Gory details of parsing quoted constructs

      General

        Exceptions are not used, there is no need for them. Please see also:

          doc/concepts.txt

        In general, the code *must not* use any additional libraries. We know
        that there exist infinite marvellous libraries and frameworks (called
        modules in Perl), which would make some programming simpler,  but one
        of the main goals of this tool is that it  should work on  any system
        with just the core language (i.e. perl) installed. We do not want any
        additional dependency, in particular no dependency on versions beside
        the core language.  Currently some Perl modules are an exception, and
        will be removed in future, if possible.
        However, libraries part of perl5 are assumed to be "core language".

        Because of our goal to support running on old systems too (see above)
        parts of the code make ugly checks for version numbers  and try to do
        checks in a cumbersome way.
        Keep in mind: the code knows how to do the work, the user most likely
        does not know.

        Perl's die() is used whenever an unrecoverable error occurs. The text
        printed will always start with '**ERROR: '.
        Warnings are printed using Perl's warn() function and the text always
        starts with '**WARNING: '.

        All output is written to STDOUT. However Perl's die() and warn() will
        write on STDERR.  Only debug messages inside $0 are written to
        STDERR.

        All  print*()' functions write to STDOUT directly.  They are slightly
        prepared for using texts from  the configuration (%cfg, %checks),  so
        these texts can be adapted easily (either with  OPTIONS  or in code).

        Calling external programs uses  qx()  rather than backticks or Perl's
        system() function.  Also note that  round brackets are used insted of
        slashes to avoid confusion with RegEx.

        Even security of the tool itself is not important (see help.txt), for
        example command injections, the parameters for qx() should be checked
        and documented why it is considered secure.

        The code flow often uses postfix conditions, means the  if-conditions
        are written right of the command to be executed. This is done to make
        the code better readable (not disturbed by conditions).

        The code is most likely not thread-safe. Anyway, we don't use them.

        For debugging the code, the  --trace  option can be used. See  DEBUG
        section below for more details. Be prepared for a lot of output!
        In fact it's a mix of tracing and debugging, to be technical correct.

      LHS condition checks

        It's a common (human) error to write assignments where conditions are
        required, in paricular in  if(), while()  and  until()  statements.
        To avoid such unintended statements, which often happen when changing
        the code,  the conditions are written with the constant  LHS  and the
        variable  RHS, for example:   if (0 < $var) { print "$var > 0"; }
        This works with constants (integers, strings) and variables only, but
        not when matching RegEx (probably with Perl 6.x).

      Comments

        Following comments are used in the code:

          # TODO:       Parts not working perfect, needs to be changed.
          # FIXME:      Program code known to be buggy, needs to be fixed.
          # NOTE:       Special brief descriptions.
          #!#           Comments not to be removed in compressed code.
          #?            Description of sub.
          #|            Code sections (documents program flow).
          ##            Comments used by third-party programs  (for example:
                        usr/gen_standalone.sh, perlcritic).
          # sub-name    Name of sub behind the closing bracket of the sub.

        Comments usually precede the code line(s) or are placed at end of the
        code line which they belong too. If the comments are placed after the
        code line which they belong too, the lines are idented.

      Annotations

        To reduce the huge amount of comments, and to write them only once if
        needed multiple times,  an annotation section  have been added at the
        end of some files. Such an annotation is written in POD format, It is
        referenced like:

            # SEE Title of Annotation

        For more details  SEE Annotations, Internal Notes (in o-saft.pl)

            perldoc $0

        This helps to make the control flow of code  more readable by humans,
        but keep the full documentation.

      Modules

        I.g. using modules, Perl modules and private ones, should be avoided.
        Unfortunately maintaining huge code  without proper modularization is
        difficult. Hence Perl modules are reduced to a minimum needs, and all
        private modules are loaded when needed only.

        All private modules are in the sub-directory  ./lib.  The name of the
        files and the module use a leading 'O', for example 'OCfg.pm'.

        While  lib/SSLinfo  uses  Net::SSLeay(1), $0 itself uses only
        IO::Socket::SSL(1). This is done 'cause we need some special features
        there. However,  IO::Socket::SSL(1)  uses  Net::SSLeay(1)  anyways.

      Variables

        As explained above, global variables are used to avoid definitions of
        complex subs with various parameters.

        Most subs use global variables (even if they are defined in main with
        my).  These variables are mainly: @DATA, @results, %cmd, %data, %cfg,
        %checks, %ciphers, %prot, %text.

        Variables defined with  our can be used in  OTrace.pm  and  OUsr.pm.

        For a detailed description of the used variables, please refer to the
        text starting at the line  '#| set defaults'  in  o-saft.pl.

      Function names

        Some rules used for sub names:

          check*        Functions which perform some checks on data.
          print*        Functions which print results.
          get_*         Functions to get values from internal data structure.
          _<function_name>    Some kind of helper (internal) function.
          _trace*
          _y*           Print information when  --trace  is in use.
          _v*print      Print information when  --v  is in use.

        Function (sub) definitions are followed by a short description, which
        is just one line right after the  sub  line.  Such lines always start
        with  '#?'  (see below how to get an overview).

        Subs are ordered to avoid forward declarations as much as possible.

        In general all names are in lower case. Using _ instead of camel case
        is prefered. However, there are some exceptions for wrapper functions
        for  Net::SSLinfo  and  Net::SSLeay .

        There're no strict rules how to construct function names. For example
        "object_action", "object_subobj_action" or "action_object". Currently
         (2024) we tend to prefer "object_action".

      Code information

        Examples to get an overview of Perl functions (sub):
          egrep '^(sub|\s*#\?)' $0

        Examples to get an overview of programs workflow:
          egrep '(^#\|\s|\s\susr_)' $0

        Example to get Perl's variables for checks:
          $0 +check localhost --trace-key \
          | awk -F'#' '($2~/^ /){a=$2;gsub(" ","",a);next}(NF>1){printf"%s{%s}\n",a,$2}' \
          | tr '%' '$'

        Makefile.dev contains following targets for easy use:
          make testarg-dev-grep_subs
          make testarg-dev-grep_sub
          make testarg-dev-grep_desc

      Debugging, Tracing

        Most functionality for trace, debug or verbose output is encapsulated
        in functions (see X&Function names& above). These subs are defined as
        empty stubs in $0. The real definitions are in  lib/OTrace.pm,
        which is loaded on demand when any  --trace*  or --v  option is used.
        As long as these options are not used,  $0  works without
        lib/OTrace.pm.

        Verbose messages always start with  '**INFO '.
        Debug messages always start with    '#dbx#'.
        Trace messages always start with    '#$0::'.
        Following formats are used in trace messages:
          #$0:: some data           - output from $0's main
          #$0::subfunc(){           - inital output in subfunc
          #$0::subfunc: some data   - some output in subfunc
          #$0::subfunc() = result } - result output of subfunc
        However, these rules are implemented very lazy.

        The prefix for verbose and trace message can be configured like:

          $0 --cfg-init=prefix_verbose="#VERBOSE: "
          $0 --cfg-init=prefix_trace="#my-trace:: "

        When  --trace  is used,  additional trace output  with timestamps are
        are also printed, even if no  --trace-time  was given. This is useful
        because it automatically "scopes" other output with  '{'  and  '}'.

        Other special coding  for debugging other than that in  lib/OTrace.pm
        is not provided.  It's recommended to use perl's  -d  option for full
        debugging.

        Just an example how to use Perl's debugging:

          env PERLDB_OPTS="NonStop frame=2 LineInfo=debug.out" \
          perl -d ./$0 +version

      Abstract program flow

          check special options and command (+exec, +cgi, --envlibvar)
          read RC-FILE, DEBUG-FILE and USER-FILE if necessary
          initialise internal data structure
          scan options and arguments
          perform commands without connection to target
          loop over all specified targets
              print DNS stuff
              open connection and retrive information
              print ciphers
              print protocols
              print information
              print checks

      Arguments (+commands and --options)

        As described multiple times, it should be possible to mix options and
        commands and other arguments in any order. It is also possible to use
        various formats of commands and options.
        A simple method to allow variants of  a string (command or option) is
        to match it against RegEx.  Unfortunately it is hard to use a generic
        way to parse commands and options. Perl's Getopt module would be nice
        but requires a hash with fixed keys.  Using a hash,  which conatins a
        proper RegEx for each command and option, could be done, but the code
        for such a sophisticated parser may be hard to understand.  So a loop
        over all arguments with a huge "switch" statment is implemented. Each
        "switch-case" matches a RegEx, and then assigns a proper value in the
        configuration. See the  "#{ COMMANDS"  and  "#{ OPTIONS"  sections in
        $0 .

      Program flow

        As explained in the documentation (please see +help) there are mainly
        3 types of `checks':
          +info    - getting as much information as possible about the target
                     its certificate and the connection
          +cipher  - checking for supported ciphers by the target
          +check   - doing all the checks based on +info and +cipher

        Most information is collected using Net::SSLinfo and stored in %data.
        All information according ciphers is collected directly and stored in
        @cipher_results . Finally, when performing the checks, these informa-
        tions are used and compared to expected well know values. The results
        of these checks are stored in  %checks .
        Then all information from  %data and %checks is printed by just loop-
        ing through these hashes.

        Information is just collected using  Net::SSLinfo  and then printed.
        Checks are performed on provided data by  Net::SSLinfo  and specified
        conditions herein.  Most checks are done in functions  'check*',  see
        above.
        Some checks depend on other checks,  so check functions may be called
        anywhere to solve dependencies. To avoid multiple checks,  each check
        function sets and checks a flag if already called, see  $cfg{'done'}.

      Documentation

        Documentation for the project consist at least of these parts:

          u)  documentation for the user
              SEE Public User Documentation (in o-saft.pl).
          d)  documentation for development
              SEE Internal Code Documentation (in o-saft.pl).
          m)  documentation for testing in development
              This dcumentation for testing the tools is done in t/Makefile*.
              There exists  t/Makefile.pod,  which contains the "Annotations"
              for the make system.  The description of the Makefile's targets
              is done as target in each Makefile itself.

        For more details  SEE Annotations, Internal Notes  and/or
        SEE Note:Documentation (in o-saft.pl), or use:

            perldoc $0

        Note that many documentation (user and development) is actively  used
        by the program itself and by some additional tools, see for example:
            $0 --help=opt
            $0 +test

        Documentation for developers is provided in various ways,  and can be
        found in:

          * the files itself:
              comments there
              annotations

          * with:
              perldoc $0
              $0 --help=test
              $0 +test
              perldoc t/Makefile.pod

          * using:
              make
              make help.doc

        (feel free to use your tool of choice instead of perldoc)

        Using make for development uses additional external tools and/or Perl
        modules:

          * dot, dotty, xdot, graph-easy

          * perl-analyzer
            (also requires Perl modules, JSON, Text::MicroTemplate)

          * Debug::Trace Devel::Trace Devel::DProf Devel::NYTProf


        Initially all user documentation was written in  Perl's POD  format.
        After two years of development, it was observerd that POD was not the
        best decision, as it makes extracting information from  documentation
        complicated, sometimes. Using POD is also a huge  performance penulty
        on all platforms. Hence  POD  will now be genereated on request using
        the  --help=pod  option.
        However, POD is still used for Annotations (see above).

    Repository

        All code is maintained in a repository. Due to various conceptual and
        personal opinions,  the master repository used internally is based on
        SCCS/CSSC, while the public repository is git/github.  The maintainer
        keeps both in sync.  The public repository must contain workable code
        only, well nobody is perfect ...

      Rules

        * Each file has at least one string containing  the unique id  of the
          file version in the repository. This id is named  SID  or SID_<id>.
          This string looks like:
            filename ID_in_repository date_of_last_change time_of_last_change
          example:     filename 1.42 18/10/23 23:42:23
          If this is assigned to a variable, it may look like:
            my $SID = "coding.txt 3.8 24/08/18 13:40:55";
          (Note beside: previous example of $SID is the SID of this file)

          Exceptions: some files for documentation only miss the SID string.

        * Changes are made in very small chunks,  each commited separately to
          the repository.

        * Different types of changes (see Commit Comments  below) must not be
          mixed in one commit.

      Commit Comments

        * Since 11/2018 all comments are prefixed with a 2-letter code, which
          describes the type of the committed change. The types are:

          Bd: - bugfix in internal (developer) documentation
          BD: - bugfix in public/user documentation
          BT: - bugfix in core SSL/TLS check or test functionality
          BF: - bugfix in general functionality or feature
          ED: - enhancement/improvement in documentation
          ET: - enhancement/improvement in core SSL/TLS checks or tests
          EF: - enhancement/improvement in general functionality or feature
          ND: - new documentation
          NT: - new core SSL/TLS check or test functionality
          NF: - new general functionality or feature

          Lower case b instead of B means  that the bug was  introduced in a
          recent change.

VERSION

        @(#) $VERSION

AUTHOR

        18. Jan. 2018 Achim Hoffmann
