Look.about Documentation for Look - LDAP operational Orca "k"ollector v. 05.05.31.01 May, 2005 Author(s): Brendan Bellina (bbellina@alumni.nd.edu) Contributor(s): perl-ldap Mods: Todd Piket, Michigan Tech, (tcpiket@mtu.edu) Sun Directory Server 5.2 default log format mod: Keith Bucher, University of Oregon, (kbucher@uoregon.edu) This file accompanies the Look.pl Perl script. This documentation is valid for Look.pl version(s): 1.10 -------------------------------------------------------------------------- Document History: v. 05.05.31.01 Addition of Sun DS 5.2 default log mod for Look 1.10 release Updated copyright notice to 2005 (better late than never) v. 04.04.30.01 Minor changes in preparation for Look 1.0 release v. 03.03.13.01 Created a version of Look.pl that uses Graham Barr's perl-ldap aka Net::LDAP for a "pure Perl" implementation. (tcpiket@mtu.edu) v. 03.02.19.01 Version format changed to YY.MM.DD.## rather than MM.DD.YY.## Added Document History section Revised Licensing section for NMI release Added notation that Look has not been tested with versions of Perl prior to Perl v5.6.0. v. 02.10.25.01 Initial release with Look v 0.90 (bbellina@alumni.nd.edu) -------------------------------------------------------------------------- Overview: LDAP operational Orca "k"ollector - Look The Look.pl script gathers LDAP performance data at periodic intervals and generates a summary file that is compatible with the open source Orca web graphing product (). Developed with funding from the Internet2 Middleware Initiative Licensing: Copyright © 2003, 2004, 2005 Brendan Bellina. All Rights Reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistribution of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistribution in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. This software is provided "AS IS", without a warranty of any kind. ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY EXCLUDED. BRENDAN BELLINA ("THE AUTHOR") AND ANY PRIOR OR CURRENTLY AFFILIATED LEGAL ENTITIES, ORGANIZATIONS, OR INSTITUTIONS SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES. IN NO EVENT WILL THE AUTHOR AND ANY PRIOR OR CURRENTLY AFFILIATED LEGAL ENTITIES, ORGANIZATIONS, OR INSTITUTIONS BE LIABLE FOR ANY LOST REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS SOFTWARE, EVEN IF THE AUTHOR AND ANY PRIOR OR CURRENTLY AFFILIATED LEGAL ENTITIES, ORGANIZATIONS, OR INSTITUTIONS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. You acknowledge that this software is not designed or intended for use in the design, construction, operation or maintenance of any nuclear facility. Requirements: Look.pl is written for Perl 5.6.0 and makes use of either PerLDAP or perl-ldap (Net::LDAP) to query LDAP. perl-ldap is recommended and is available at . Command Line Options: Command line options should be prefixed with "-" and can be entered by single initial or fully specified name. If an option takes a parameter then the parameter should immediately follow the option. -usage Displays command line usage rather than executing the script. This option does not take parameters. -debug If present, then debug mode is turned on. This option overrides the debug user preference. This option does not take parameters. -windows Forces a bypass of the already-executing check. Handy if running in the Microsoft Windows environment. This option does not take parameters. -nofile Redirects output to the console. Handy when debugging or adding new statistics columns. This option does not take parameters. -binddn Bind dn to use when querying for LDAP stats. This option overrides the binddn user preference. This option takes a required single parameter. -passwd Password to use with the binddn when querying for LDAP stats. This option overrides the bindpasswd user preference. This option takes a required single parameter. -altpref Path and name of an alternate user preferences file to use rather than the default "Look.up" in the current directory. This option takes a required single parameter. User Preferences: The user preferences file for this script should be stored in the same directory as the script and should be named "Look.up". The -altpref command line option can be used to override the path and name of the user preference file. The user preferences file is standard text. Parameters are tab and newline delimited. Each parameter should begin in column 1 of a newline and be spelled exactly as shown below. Blank lines and lines whose first character is # are ignored. sysparmfile An optional parameter for the path and name of a system parameters file. If present then the system parameters file will be read. Locally defined parameters override system-defined parameters. debug set to 1 to turn on debug mode, 0 to turn it off. debug mode can also be turned on using the -debug command line parameter which overrides the user preference setting. logprocess_flag set to Y if log file harvesting should be done, N if not. Either logprocess_flag or ldapprocess_flag must be set to Y, or both. ldapprocess_flag set to Y if direct queries to the LDAP server for stats should be done, N if not. If Y then the ldapport, ldaphost, and ldapmonitor parms must also be set and you need to have PerLDAP or perl-ldap installed. Optionally the ldapmonitorldbm parm may also be set. Either logprocess_flag or ldapprocess_flag must be set to Y, or both. ldapprocess_cont set to Y if ldapprocess_flag is Y and you want processing to continue in the event of LDAP errors. Useful if you know that there will be times that your LDAP server will be unavailable and you want processing to continue. Only relevant if using LDAP processing (ldapprocess_flag = Y). ldapport The port to use for LDAP queries when performing LDAP processing. Only required if using LDAP processing (ldapprocess_flag = Y). Normally this is 389 for cleartext LDAP and 636 for LDAP over SSL. ldaphost The LDAP host name. Only required if using LDAP processing (ldapprocess_flag = Y). ldapversion The LDAP version to use (2 or 3). Only required if using LDAP processing (ldapprocess_flag = Y). Only required if using the perl-ldap version of Look.pl. ldapmonitor The LDAP dn to use when retrieving stats from the LDAP directory via query. Only required if using LDAP processing (ldapprocess_flag = Y). ldapmonitorldbm The LDAP dn to use when retrieving ldbm stats from the LDAP directory via query. Only relevant if using LDAP processing (ldapprocess_flag = Y). collection_interval Specifies in seconds how often data is collected and over what period peaks are determined. collections_per_write Determines after how many collections totals are written to the Orca output file. rotate_out_time Specifies what local time to rotate the Look.out log. Specify time in format hh:mm By default the rotation time is 23:30 (11:30 pm). ldap_logfile The full path and name of the log file to process. Only required if using log processing (logprocess_flag = Y). logtype A code that indicates the format of the log file to process. Only required if using log processing (logprocess_flag = Y) Current valid logtype values are: Code Type of log file ---- -------------------------- iDS4 iPlanet Directory Server 4 (compatible with SunONE 5 logs) sunDS52 Sun Directory Server 5.2 default logfile format outputfile The path and name of the Orca-formatted file to write output to. Each time the output file is rotated a datetime stamp will be appended to the filename. The command line parameter -nofile causes output to be written to standard out rather than a file. The recommended file name is "Look.out". debugfile The path and name of the file for debug output when debug mode is selected (either by setting debug to 1 or specifying the command line option -debug). The command line parameter -nofile causes debug output to be written to standard out rather than a file. The recommended debug file name is "Look.dbg". processcmd The command used to determine if the process is already executing at initiation. The already executing check is bypassed if command line option -windows is specified. calcavgpeak_flag Set to Y if average peaks should be calculated, N if not. ldapmonitorbind Set to Y if an authenticated bind is required to retrieve directory stats, N if not. Relevant only when ldapmonitor = Y. If set to Y then binddn and bindpasswd should also be set to valid values. binddn dn to use for binding to the directory when ldapmonitor=Y and ldapmonitorbind = Y. Required only if ldapmonitor = Y and ldapmonitorbind = Y. The command line parameter -binddn can be used to override this value. bindpasswd Password to use for binding to the directory when ldapmonitor = Y and ldapmonitorbind = Y. Required only if ldapmonitor = Y and ldapmonitorbind = Y. The command line parameter -passwd can be used to override this value. ldapmonitorSSL Set to Y if SSL should be used when retrieving directory stats, N if not. Relevant only when ldapmonitor = Y and ldapmonitorbind = Y. If set to Y then SSLcert must also be set to a valid value. SSLcert Path and name of the certificate database. Required only if ldapmonitor = Y and ldapmonitorbind = Y and ldapmonitorSSL = Y. SSLVerify Tells the SSL client to verify the authenticity of the certificate presented by the server. Acceptable values: required none The required value tells the client to check authenticity. The "none" value tells the client to ignore authenticity check. Required only if ldapmonitorSSL = Y. Only required if using the perl-ldap version of Look.pl. Support: While email can be sent to the author, there is no promise of support. The user takes full responsibility for any effects, positive or negative, of this software. To the positive, there is also no official support fee. Suggestions and feedback are welcome and can be directed to the author at bbellina@alumni.nd.edu. Installation: Perl5 is required. (Versions prior to 5.6.0 have not been tested and may not work). PerLDAP or Graham Barr's perl-ldap (Net::LDAP) is required if using LDAP processing. You may need to comment out some code if neither is not installed. Orca is required. Orca is available at . ******************************* There are two scripts in the Look distribution. Look.pl.mozilla uses PerLDAP and Look.pl.perl-ldap uses Graham Barr's perl-ldap implementation. It is recommended you choose the one that suits your environment and rename it to Look.pl as the rest of this documentation refers to the Look.pl script. Future enhancements are expected to be implemented in the perl-ldap version only, since PerLDAP is no longer an active product. ******************************* The Look.pl script assumes its configuration file is in the current directory and is named Look.up. This can be overridden with the command line option -altpref. Orca configuration required to support Look: Orca uses a configuration group called orcallator. This group has a find_files line which needs to be modified so that the Look.out files are recognized. The Look.out files look like "Look.out" with an optional "." + perl datetime stamp, so the find_files parm should be changed to resemble: /opt/orca/var/orca/Look/orcallator/(.*)/Look\.out(?:\.\d{14})? The find_times option in the Orca configuration file should be set to a range of times around the time that you have specified for the Look rotate_out_time preference. (A range of 10 minutes before and after would be reasonable, but some experimentation might be required.) The Orca configuration file needs to be modified to include the graphs that you wish to create. The Orca documentation explains the graphing options, but to provide some assistance, below is the text for the graphs used by the author: (see the currently plotted graphs at ) plot { title %g LDAP connection rate source orcallator data connops/s data SSLconnops/s data connops/p30s data SSLconnops/p30s color 00ff00 color ff0000 color 0000ff color ff00ff line_type area line_type stack line_type line1 line_type line1 legend Conn_5m legend SSL_5m legend Conn_30s legend SSL_30s y_legend ops/s data_min 0 plot_min 0 } plot { title %g LDAP BIND rate source orcallator data anonbindops/s data bindops/s - anonbindops/s data anonbindops/p30s data bindops/p30s - anonbindops/p30s color 00ff00 color ff0000 color 0000ff color ff00ff line_type area line_type stack line_type line1 line_type line1 legend Anon_5m legend Auth_5m legend Anon_30s legend Auth_30s y_legend ops/s data_min 0 plot_min 0 } plot { title %g LDAP BIND error rate source orcallator data binderrors/s data binderrors/p30s color 00ff00 color 0000ff line_type area line_type line1 legend BindErr_5m legend BindErr_30s y_legend errs/s data_min 0 plot_min 0 } plot { title %g LDAP search rate source orcallator data srchops/s data srchops/p30s color 00ff00 color 0000ff line_type area line_type line1 legend SRCH_5m legend SRCH_30s y_legend ops/s data_min 0 plot_min 0 } plot { title %g LDAP SRCH error rate source orcallator data srcherrors/s data srcherrors/p30s color 00ff00 color 0000ff line_type area line_type line1 legend SrchErr_5m legend SrchErr_30s y_legend errs/s data_min 0 plot_min 0 } plot { title %g LDAP search results returned rate source orcallator data srchresults/s data srchresults/p30s color 00ff00 color 0000ff line_type area line_type line1 legend SrchResult_5m legend SrchResult_30s y_legend entries/s data_min 0 plot_min 0 } plot { title %g LDAP connections source orcallator data conn data conn/p30s color 00ff00 color 0000ff line_type line1 line_type line2 legend conn@5m legend conn_30s y_legend connections data_min 0 plot_min 0 } plot { title %g LDAP operations initiated source orcallator data d_opsinit/p30s data d_opsinit color 00ff00 color 0000ff line_type area line_type line1 legend ops_init_30s legend ops_init_5m y_legend operations initiated data_min 0 plot_min 0 } plot { title %g LDAP operations completed source orcallator data d_opscomp/p30s data d_opscomp color 00ff00 color 0000ff line_type area line_type line1 legend ops_comp_30s legend ops_comp_5m y_legend operations completed data_min 0 plot_min 0 } plot { title %g Entries sent source orcallator data d_entrsent/p30s data d_entrsent color 00ff00 color 0000ff line_type area line_type line1 legend entr_sent_30s legend entr_sent_5m y_legend entries sent data_min 0 plot_min 0 } plot { title %g Bytes sent source orcallator data d_bytesent/p30s data d_bytesent color 00ff00 color 0000ff line_type area line_type line1 legend byte_sent_30s legend byte_sent_5m y_legend bytes sent data_min 0 plot_min 0 } plot { title %g Entry Cache Hits source orcallator data d_entrchits/p30s data d_entrchits color 00ff00 color 0000ff line_type area line_type line1 legend entry_cache_hits_30s legend entry_cache_hits_5m y_legend entry cache hits data_min 0 plot_min 0 } plot { title %g Entry Cache Tries source orcallator data d_entrctries/p30s data d_entrctries color 00ff00 color 0000ff line_type area line_type line1 legend entry_cache_tries_30s legend entry_cache_tries_5m y_legend entry cache tries data_min 0 plot_min 0 } plot { title %g Cache Hit Ratio source orcallator data chitratio data chitratio/p30s color 00ff00 color 0000ff line_type line1 line_type line2 legend c_hit_ratio@5m legend c_hit_ratio_30s y_legend cache hit ratio data_min 0 plot_min 0 } plot { title %g Database Cache Hits source orcallator data d_entrdbchits/p30s data d_entrdbchits color 00ff00 color 0000ff line_type area line_type line1 legend db_cache_hits_30s legend db_cache_hits_5m y_legend database cache hits data_min 0 plot_min 0 } plot { title %g Database Cache Tries source orcallator data d_entrdbctries/p30s data d_entrdbctries color 00ff00 color 0000ff line_type area line_type line1 legend db_cache_tries_30s legend db_cache_tries_5m y_legend database cache tries data_min 0 plot_min 0 } plot { title %g Database Cache Hit Ratio source orcallator data dbchitratio data dbchitratio/p30s color 00ff00 color 0000ff line_type line1 line_type line2 legend db_c_hit_ratio@5m legend db_c_hit_ratio_30s y_legend database cache hit ratio data_min 0 plot_min 0 } Initiation Instructions: Look.pl can be initiated via the UNIX command "Look.pl &" as long as the initial line of the script points to Perl5. If your Perl modules are not in the default Perl location you may need the -I parameter on the first line as well. The process will use the command specified in parm processcmd to determine if it is already executing, and if so will fail to initiate. This check can be bypassed using the -windows option. If you want Look.pl to start upon machine startup then you may want to implement a start up script. ----------- End of file