MW-E2ED Conference Call
October 4, 2006

*Participants*
Chas DiFatta, CMU (chair)
Paul Hill, MIT
Mark Poepping, CMU
Steve Olshansky, Internet2

*Discussion*
Chas reported a new version of EDDY was released in early August. The release includes a lot of network NetTalkE functionality and new CERs. In addition, the documentation has grown to almost 200 pages. The release has prompted more interaction with the folks at the University of Washington.

Chas talked with UWash about EDDY's feature roadmap and what their priorities might be for the next release (which will called Murphy). One of the things they prefer is a more robust logging internally to the EDDY framework or backplane. That was on our docket to do. We worked with them in August and adopted Log4j. The EDDY backplane will log to Log4j and this will be included on the next release. Also planned are more formal specifications for CERs with an exhaustive explanation of what every field does.

Going forward, there are a large set of features to be incorporated into EDDY. The challenge is determining what to do next. Chas referred to the feature roadmap, wihich is part of the 2006-07 development timeline on the EDDY web site: http://www.cmu.edu/eddy/software-dist.htm

The goals of the 2006/2007 development timeline will be,

• Add internal fault and management event capability to EDDY framework  This is about half done – it is at the point where we can do some debugging.
• Provide an Email diagnostic toolset. CMU has provided additional resources for this – currently working on diagnostic tool for email problems. Based on Paul/Michael suggestion, developing a system that extracts syslog, spam engines, etc, and placing them into the database. A nice front end up and running and we're now tweaking everything, especially the data model.
• Include Toolkit for Security Researchers
• Simplify the adoption of the framework by requiring less Java programming expertise as well as an event visualization suite
• Expand transport functionality of the event channel to a publish/subscribe model and implement the query and control channels
• Continue to increase the functionality of the network diagnostic toolset
• Add additional events into the CER framework

Core Framework Feature Candidates
• Plug in transport mechanism - enable agents to subscribe/publish to an event stream without having to change the configuration file.
• Internal diagnostics and management - improve internal logging of framework errors (implement log4j) and produce two new types of CERs that each agent would be required to produce. One a summary CER to be produced on a regular interval and two, an internal error CER
• Provide an Email diagnostic toolset
• Improved certificate strategy - specify a certificate store as well as a password on configuration
• High availability/fail over of streams - provide a mechanism for fail over to a secondary path for the event channel
• Control channel - implement mechanism for controlling and configuration an EDDY cluster via a web interface and communication to all agents within the cluster via the management agents
• Query channel - implement the query channel (as well as discovery and directory features) via SOAP to enable the retrieval of events from storage agents
• agent manager - tighter integration between present agent manager and agents
• filtering - replace RPN based filtering functionality with a more traditional expression evaluation
• filtering (type and context based) - filtering option in the configuration that allows for XML marshaling and raw for high performance event flows

Additional Normalizer Candidates
• Snort events
• Cisco NetFlow (9)
• Syslog – getting a handle on right now. Right now not changing syslog events to CER, but are working toward that.
• BGP events
• LDAP
• DNS
• Web events (Microsoft Explorer and Apache)
• Generic Environmental Sensor Events (temperature, light, sound, humidity, etc.) – this is pretty much done.
• Mail logs (sendmail, Cyrus, etc.)
• Microsoft event logs
• Microsoft MOM (Microsoft Operational Manager) import/export
• Email event normalizers (POP, IMAP, SMTP, spam engines)
• Add more formal structure to the user tag fields – have made some changes. Anyone can add any specific tag to any CER that exists.
• Design and implement a general lightweight CER to be used for experimentation
• Argus 3.0 (a flow generation tool)
• EDDY TopNetPorts - profiling of network services. Interest from folks in security office. They use TopNetE and they would like a similar tool that provides the average port flows with respect to bytes, network flows or packets on a specific destination port, based on an average with a standard deviation. Want this to see trends (in several time periods: one hour, eight hours, one day, one week, and two weeks).

Workbench - agents for building diagnostic applications
• Generic storage agent - general purpose storage agent that implements the query channel and stores events on disk. Sustainable event capture should be less then 1000 events/sec.
• High performance storage agent - high performance storage agent that implements the query channel and stores events on disk. Sustainable event capture should be greater then 20000 events/sec.
• CER factory - provide a simple service that could read a CER template and generate an official CER. Provide also a simple Perl interface for submission and subscribing. Possibly use SOAP as an interface.
• Anonymization - rework of the present 0.6 anonymization agent to make it more general purpose. Now focused on  IP addresses. Want to make that more generic.

Diagnostic Applications
• Focus on solving problems in the EMail domain from the perspective of the mail administrator and help desk
• Security focused, combing generic network flow data with Snort
• Include network flow detail into the TopNetE diagnostic tool – filtering on destination port numbers. Click on IP address to get detailed information.
• Generic event visualization tools
• Security tool that measures service port trends over time

Discussion centered on Email as a good place to start, since everyone has email and it is essential to organizations. The committee proposes a presentation or demo, with these options:
• a series of screen shots
• a WebX style demo that one person can control
• creating a data set for people to manipulate
• use a video stream/narration that people can play at their own convenience, including video clips and animated diagrams

The consensus was using the latter – a video stream/narration.

*Frequency of meetings*

Meetings will be monthly – the first Wednesday of each month.

*Calendaring systems*

Chas asked for thoughts about diagnostics with respect to calendaring systems. Paul commented that an approach to getting the attention of vendors may be to share the video/demo about email diagnostics. Once that video/demo is created, he would send mail to all those in the calendar/schedule consortium, asking them to think about this technology in terms of calendaring.

Next call: Wednesday, November 1, 5:00 EST