Contributions reordered

• + - General

  • The report improved a lot (better structured). Excellent and easy to track the milestones. More uniformity and good balance between the areas.
    

      I agree very much with Wisla that the report has improved a lot in terms of cohesion of contents, clarity and format.
    
      I have problems however with the lack of longer-term milestones and, as Wisla, with the lack of real meaning of certain milestones simply stating something has started. I will come back to this more concretely at the end of this e-mail.
    
      Another important general point is that, given the noticeable time-lag between the actual submission of the LCG report by Les and the report compiled by Matthias for the POB, many of the upcoming milestones elapse within this time period (all such examples will be listed below). It would therefore be extremely useful, as has been done for some of the work-packages, that all upcoming milestones be listed with some expectation of whether they would be met or not. Les will undoubtedly update us on this Friday, and perhaps it would be useful to have the area managers present during this discussion. I do not wish to mix two milestone reports together through this, I just wish to realign today's status of the project with what the perception of it within the SC2 might be.
    

    1. General Comments.
    1.1. The Project report is much improved in clearness and readability. Thanks for the effort to the Area Managers and the Project Leader, as the following of milestones delivery and progress is much more easy.
    1.2. Overall very good progress of the Project, however still on critical path for deliverables/milestones (see below), personnel on key areas and planning on challenging activities (e.g deployment of stable structured services or common software/middleware availability).
    1.3. Project has entered a new phase of work, starting to consolidate activities. The (rapidly) changing of personnel in all the areas and sub-areas must be planned, and indeed it’s being planned as seldom reported. However this activity introduces an “overload” of needed FTEs and the issue should be estimated or exposed. It’s positive that many expertises are starting to move around the different sub-projects of the LCG challenge, as it will help, but it has to be taken into account.
    1.4. It’s becoming clear that the projects where the experiments are directly involved (personnel and interest) are progressing better than others. Deeper integration with experiments’ activities should be further pursued, as this strategy is proving to be successful (as expected).
    1.5. Many Milestones are either late or delayed (see below), even if the majority of them were met in the period. Reasons of the delay or of the lateness are correctly reported, including the possible effect of them and actions to do. However the general “trend” of lateness is a reality. Coming out from the “starting phase of the Project” this general trend must be corrected; either slipping to a better possible dates ALL the Milestones, or carefully proposing new affordable dates for each specific milestone. This point is of paramount importance for the Experiment planning and deliverables in due time for LHC startup.
    1.6. Some of the general consideration of chapter 1 of the report state key strategies and future development of the Project. Most of them are already agreed and carefully planned, and therefore help the understanding of the progress. Other is wishful thinking or not (yet) fully agreed issues, as e.g the C-RRB manpower support for “Grid Services” (there is a lot of space for contributions to infrastructure support and also “normal” experiment services”) or the collaboration with EGEE for common services deployment.
    I’d prefer the Report to stick on milestone report on the quarter and possible agreed development for the short time coming ones.
    

    General
    ======
    - all milestones must be re-addressed to allow for holiday periods - too many missed for this reason
    - impression that verification milestones need strengthening - another round with the expts
    - concentrate on porting to currently supported platform before diversification elsewhere
    

• + - AA

  • 2. Application Area. FTEs: 53.7 unweighted (few FTEs less than previous report).
    Nothing general from my side to add to the reports already made my colleagues of the other LHC experiments), except a couple of points:
    a) ROOT participation sub-project has NO Milestones, or direct deliverables reported or planned. Should we have some? I think yes.
    b) Core/Grid Interface & Experiment Integration is STILL (at least two QR reports since now) missing of possible integration plan with the Grid Deployment area similar (same?) activity, and is “consuming” 4.6 weighted FTEs. Need for a better LCG Project definition of responsibilities is urgent and becoming critical.
    
    It’s also anticipated that some (many?) of the coming quarter milestones of the AA area will be delayed (and we know nowadays that ARE delayed).
    

    + - SPI

    • SPI:
      The milestone concerning SPI 2003 workplan including manpower profiles had been completed in the 2nd quarter (last report said May 2003), the level of manpower was considered to be "barely adequate" , it has been as expected throughout this quarter.  Therefore one could argue why priorities have been put in other tasks (successfully achieved), such as LCG workbook, Policy and quality assurance activities,  and not in the only level 2 milestone (SPI support for Windows binary version of LCG sw) that was due in the third quarter.  The delay of this milestone is said to seriously affect LHCb. Has the new settting of the milestone (October) been agreed with the experiment? The connection of this planning with the planning of the experiments is of paramount importance.
      

        a)SPI
        --------
        No specific comments. Windows milestone missed significantly, and transition to IT/CVS expected by end of this month. Are there any problems anticipated with this?
      

      2.1. SPI (8.7 FTEs).
       Milestone 1.17 (Support for windows…) late > 2 months.
      

    + - POOL

    • POOL:
      - Very important milestones (integration with experiments sw) achieved.
      - Delay is correctly reported for the intial POOL deployment on LCG-1 (to November, maybe a bit later than the 15th, since LCG-1 is foreseen to be fully operational on the 20th).
      

        b) POOL
        ------------
        A lot of progress with the integration into CMS (mostly) and also to a lesser extent ATLAS. The perceived need to increase the level of effort in integration is well founded and certainly supported by ATLAS.
      
        However, as highlighted also by the AA review, there is a lack of longer-term milestones, which should address issues such as e.g. schema evolution on a reasonably aggressive time-scale. The only remaining milestones under POOL (apart from verification milestones from the experiments) are "POOL meets scalability requirements due 31/10/2004" and perhaps "Full function release of persistency framework due 01/03/2005" under the Physics Data management. Perhaps the milestone "2004-2005 persistency framework workplan complete due 15/12/2003" will provide new milestones for the POOL project and probably account for the outcome of the recent review and of further discussions on dictionary issues. Could this be confirmed by Torre?
      
        Is the milestone "POOL RDBMS independence layer in beta", due 30/09/03 ok?
      
        There is no mention of the important milestone "POOL implementation of conditions DB -beta" due 01/11/03. Has this been met?
      
        One last small question concerning POOL: should one consider the milestone "Initial POOL deployment on LCG-1" due 15/11/03 as meaning now "Initial POOL deployment on LCG-2" due perhaps a week or two later?
      

      2.2. POOL (12.3 FTEs).
       Milestones (three) mostly OK.
      

    + - SEAL

    • SEAL completed during the 3rd quarter (part of) the milestones due in the 2nd quarter, demoted one milestone to level 3 (since "the lack of it has not been felt") and added a new milestone to complete the first set of GSL enhancements. The project seems to be progressing well, but more attention to keep the delivery date is needed.
      

        c) SEAL
        ------------
        ATLAS supports strongly the encouragement from the review that there be much increased synergy between SEAL and ROOT. It is also very good news that LCG now hosts CLHEP, and ATLAS would encourage a more active and direct support of CLHEP (e.g. to resolve dictionary issues with CLHEP matrices), and see a clear link here between possible future SEAL support to HEPMC and WP2 in the Generator Services, on one side, and the MCtruth work in the Generic Simulation framework on the other side. Resolving the issues linked to MCtruth in general, HEPMC in particular, will require a dedicated review process and would perhaps justify a new high-level milestone.
      
        There appears further ambiguity between SEAL and PI. This had already been noted in previous SC2 discussions when going through reports on the SEAL and PI projects (in effect, the hot potato of making the objects in memory browsable to the ROOT prompt, has by now fallen into a crack between SEAL and PI). This ambiguity appears now in terms of developments for ARDA services.
      
        One way to solve this would be to query the need for a continuation of PI beyond its limited objectives agreed when it was presented to the SC2 early this year.
      

      2.3. SEAL (5.4 FTEs).
       Two milestones OK.
       Milestone 1.124 (Statement on GSL and Math libraries…) delayed of one moths (new milestone)
       Milestone 1.184 (Math work plan..) late of more than 2 months (even if work is ongoing)
       Milestone 1.187 (support for windows…) late of about two months.
      

    + - PI

    • The only milestone (Report from AIDA interface review) due in this quarter has been delayed due the slowness of users feedbacks from experiments. It remains the only milestone listed in the upcoming milestones.  
      Why doesn't the release 1.0.0 (announced in the text part for mid October) appear in the upcoming milestones?
      The planning of the PI should be reviewed according to the outcome of ARDA RTAG. A milestone to review the planning is needed.
      

        d) PI
        -------
      As discussed above, the overall future scope of this part of the project should at least be reviewed.
      
      It is hard to see how the AIDA interface review can be completed by end of October (already past) when there has been no push for nor real feedback to such a review from ATLAS yet (until we achieve significant POOL functionality within ATLAS, there will be no user base for this).
      

      2.4. PI (5.1 FTEs).
       Milestone 1.171 (AIDA interface review (users)…) late of about two months.
      

    + - Simulation

    • Generic simulation framework milestone superseded by the availability of an intial prototype. But in Appendix 2 it is said to be due for the 30 of July.
      The simulation physics requirements milestone has been delayed to December. No hint on the consequences.
      What about the EM physics validiation complete milestone? it is neither in the achieved milestones, nor in the upcoming ones. In the Appendix 2 it is said to be due for the 30th of September.
      

        e) Simulation
        -------------------
        ATLAS notes that the high-level design milestone on the generic framework is dropped basically because of the still fluid situation with the evolution of this part of the project and expresses real hopes    that the more concrete milestone replacing it, namely "Generic simulation framework prototype available (G4 and Fluka)"  will meet its deadline of 31/12/2003.
      
        It is rather sad to see the sentence "New requirements from CMS regarding direct linkage of simulated particle tracks with the primaries from an event generator ..", when this fundamental requirement which is plain common sense had been forgotten about in the original requirements and requests from ATLAS for the past two years have met until now with little or no success. In any event, thanks to CMS and to the BABAR G4 developers!
      
        Again, both Fluka and G4 should have new milestones linked to the generic problem of MCtruth (together with HEPMC) and progress in this area should be reported through the Generic Simulation Framework part of the project.
      
        As mentioned above, the issue of HEPMC as primary interface to generators and as one important element of MCtruth should be addressed with the participation of the authors of the packages contributing to GENSER. This service is currently under evaluation by ATLAS and progress appears to have been good.
      
      A few smaller points to be considered:
        - the milestone "Simulation physics requirements revisited for all experiments" has not really been completed for ATLAS (for example, we are still struggling to define precisely our MCtruth requirements).
        - the first cycle of EM physics validation is reported as complete but not mentioned in the text?

      2.5. Simulation (16.0 FTEs).
       Milestone 1.144 (Generic framework high level design…) postponed to end December 03 (was end of July in the previous quarter report!): late more than 3 months.
       Milestone 1.14 (Simulation physics requirements revisited) postponed by more than 5 months (ATLAS completed however), December 03.

    SPI: continuing delays with Windows is serious. Effort need to be put into porting of SCRAM from the experts.
    POOL: important that functionality improvements are frozen to allow time for performance testing
    SEAL: still need to see math libraey evaluation report - this is important.
    Concerned over delay of external s/w  adoption decisiom making process - particularly relevant for outside of CERN - not  acceptable to downgrade milestone
    PI:  awaiting ARDA
    Simul: note concern on generator support (which seems relating to development of a framework rather than just librarian work); adoption of LHAPDF without discussion of interfacing to generators; should note new UK resources may be available for to develop JetWeb (decision should be known soon)
    

• + - FA

  • Milestones achieved. Progressing well.
    A bit worring the huge number of upcoming milestones in the next quarter, some of which are already anticipated as delayed because of the the LCG-1 delay. No mention whether the available manpower is adequate.
    A minor point: I am not in favour of having milestones of the kind 1.2.2.7 Development started - since what we are interested in is not really when the development has started, but when it is available.
    

      II) Fabric
      =======
      Progress has been excellent, as shown recently from direct report to SC2. The written report does not demonstrate this as clearly, perhaps, as pointed out by Wisla, because there are many milestones which should be removed since they have little concrete significance (this or this has started ...).
    
      Most milestones due in the next quarter have already elapsed and there is no indication in the report whether these are likely to be met. This should be remedied in the future, as suggested in the general introductory comments.
    

    
    3. Fabric Area. FTEs: 41.3 unweighted (few FTEs more than the previous quarter report).
    I’d like to stress again (since two reports now) that there is still some lack of “outside CERN” cooperation that could help the Project wide up.
    QUATTOR is a good news, however (again) how it interfaces with other “similar” projects “outside CERN”?.
    3.1. Milestones
     Milestone 2.532 and 2.552 are start of activities not real milestones, however OK.
     Milestone 2.538 (SPMA full prod) late of 18 days, good.
     Milestone 2.552 (Right half of m/c room migrated…) late of 15 days, good.
     Milestone 2.562 (or WBS 1.2.4.2.1) missing in the report (10 Lxbatch integrated in LCG-1… due the 01-08-03). I can imagine why, however should be reported.
    In addition there are at least three milestones of the coming quarter that are anticipated to be delayed by more than few months (WBS 1.2.1.2.1, 1.2.4.2.2 and 1.2.4.2.3)
    

    - concern about the situation with RH - need plans in place with high priority

• + - GTA

  • Relationships with GDA - EGEE still partially unclear (to me at least). There are anticipated delays in the upcoming milestones, which will become responsibilites of the joint LCG/EGEE Middleware Manager, while the long-term middleware strategy will stay within the GTA.  Without a new plan and new milestones taking into account the changes in responsibilities, a real analysis of the milestones in the GTA area is impossible, mainly because the future work is mostly
    

      III)  GTA
      ======
      In this transition phase between GTA as it was and EGEE work integrated into GTA as it will be, it is not surprising that the report looks strange (e.g. resources close to nil and probably unrealistic milestones for  next quarter). What would the revised milestones look like (I assume they are all linked to the setting up of the EGEE team)? What will be the role of the current GTA manager in the future (although some SC2 members may know the answer to this, I have not yet heard a clear unambiguous one).
    
    

    4. Grid Technology Area. FTEs: 5.0 unweighted (don’t know bout previous quarter)
    Discussion and re-evaluation of the area scope are going on in relation with Grid Projects, so little can be said about general points.
    4.1. Milestones:
     One milestone OK (3.337, modeling…)
     Milestone 3.344 (GT3 report) late of ~ 1 month
     Milestone 3.501 (EGEE senior manager…) late of more than 2 months
     Milestone 3.502 (technical design team…) delayed awaiting ARDA and HEPCAL II, however late more than 2 months.
    In addition there is at least a milestone anticipated to be late for the next quarter: 3.503 (2004 blueprint architecture), even if also waiting for ARDA and HEPCAL II reports.
    

    - awaiting ARDA
    - establishment of tech development team should be given top prority
    

• + - GDA

  • All milestones of 3rd quarter met, even if with an average delay of 3 months.  Next quarter milestones still have unrealistic dates (such as experiment verification of acceptance testing of LCG-1 for the 1st of
    October, middleware functionality complete for the 1st of October, etc). Wouldn't it be more realistic to move future milestones that we already know as impossible to be met, due to the LCG-1 delay (as it has been done for example with the initial POOL deployment on LCG-1, correctly moved from July to November 2003)?
    

      IV) GDA
      =======
      I agree fully with Wisla that some cleaning up is needed here,  both in terms of naming conventions (there exists an LCG-2 now), and in terms of real milestones given the large delay to the deployment of LCG-1 with its close to full functionality (now called LCG-2?)

    5. Grid Deployment Area. FTEs: 25.5 unweighted (few FTEs more than the previous report).
    The deployment is entering the “running” phase, and so is progressing well. We have still to wait for assessment, however the rising speed is promising.
    5.1. Milestones:
     Milestone 4.353 (Initial middleware for LCG-1…) late of 3 months.
     Milestone 4.355 (Deploy LCG-1 m/w to 10 Tier1 sites) late of more than 2 months.
     Milestone M1.1 (First global service…) late of about 2 and half months.
     Milestone 4.956 (LCG-1 certified and commissioned) late of more than 2 months.
     Milestone 4.957 (LCG-1 performance…) OK.
    
    In addition there are at least 5 milestones of the coming quarter (out of 9 in total) that are anticipated will be delayed by more than 2/3 months.
    
    

    - concern about development of "lite" deployment system - should be given high priority - poss. even more relevant when going to Tier-2 centres. Part of deployment problem at some of original Tier 1's
    - note LHCb yet to test LCG-1 (In process of packaging LHCb s/w in format suitable for LCG-1)
    - concern at plans for GFAL & interfaces to MSS. Only mention to FNAL & CERN - committments from other centres essential
    - no discussion of resources available in production service in January