Subject: Post EDG 2.0 planning at PTB From: Erwin Laure Date: Mon, 25 Aug 2003 11:28:59 +0200 To: WP MGR Hi, Thanks everybody for the input to the PTB discussion. You find the collected input below. All of the post 2.0 integration work needs to respect the following boundary conditions according to the original TB3 release plan (https://edms.cern.ch/document/382573): - support for 2.0 has highest priority intense testing of 2.0 will continue on the application TB as well as by LCG and bugs and problems found need to be tackled with highest priority. - all software needs to be stored in the unified CVS repository (Lyon). - all software needs to be delivered via autobuild (only gcc 3.2.2). - the release procedure defined in the original TB3 release plan needs to be firmly adhered to. The first steps before integrating new components will be: - complete port to gcc 3.2.2 - VOMS server setup (and testing for at least 1 VO) Assuming we can deploy 2.0 this week and finish the gcc 3.2.2 port and VOMS server setup by the end of next week (5th September) this leaves us only 3 weeks until the begin of the Heidelberg conference, hence only two integration slots. What will be integrated at these two slots needs to be discussed at the PTB based on the input you find below. All the other work that will not be integrated into the final EDG release should be appropriately reported in the deliverables and may be demonstrated on the WP testbeds at the review. Cheers, -- Erwin ------------ WP 1 ---------------------------- From: Francesco Prelz We still plan to get to the final release with DAG submission in place, which we plan to show at the final review. This does not mean that all what you mention is ready, because we know far too well, and have learnt and are learning time and again, that the scale factor is hard to really 'test' against. This is why we are taking the unique chance to handle the precious and accurate feedback of LCG and keep linting the code. At any rate, I haven't had a chance to further discuss within WP1 after our last planning meeting at the beginning of July, and I'm basically all by myself now. We'll be glad to reah consensus about this at the PTB. ------------ WP 2 ----------------------------- From: "Gavin McCance" RLI integration. RLI service all done and ready. The replica manager integration is coded in our HEAD branch but needs full testing. java-security trustmanager and tomcat: All ready, few random non-critical bug fixes remaining in edg-java-security. Authorization work. 2 weeks to get one service running. 1 week after to get the others going. ------------------ WP 3 ------------------------------- From: Steve Fisher All of the R-GMA code has already been tested with the gcc3.2.2 compiler and is course all in the official CVS. Some of the components promised for 2.1 are already in 2.0: - More functionality in the mediator to handle joins when all relevant tuples are in one RDBMS for Latest or History queries. - the registry replication code is part of the 2.0 release but is disabled as we have only given it one registry in the configuration file We will be able to deliver the following: - Registry replication This has been well tested by unit tests and manual testing over a group of machines and is documented (only a few lines about the configuration file). LCFG is done but we need to test further by deploying on the WP3 testbed. - Authentication This has been also been well tested by unit tests and manual testing over a group of machines and is documented both in our user guide and the installation guide. But we still need to complete the LCFG work. - A new version of GRM using the mercury monitor This has been also been tested by manual testing over a group of machines and is documented. From a user perspective it looks like the old GRM but is much faster. The LCFG work is almost complete. We will not be able to deliver: - schema replication. - other mediator improvements - making use of VOMS In addition, and independent of the release plan as it need only be run under one node under our control: - Nagios/R-GMA integration So to summarise authentication; registry replication and the new GRM are all complete except in some cases completing the LCFG configuration and making sure that we get no nasty surprises on the WP3 testbed. If either the replication or security give problems they can easily be disabled - though we do not foresee any problems. -------------------- WP 4 ------------------------------ From: "Maite Barroso Lopez" Here you have the list of WP4 components we would like to integrate for EDG 2.1 together with the realistic date for completion (tests, test report, autobuild...): -LCMAPS framework + plugins (VOMS module, POSIX module, POOL module): ready for integration -LCMAPS AFS and Kerberos modules: ready end of September -LCAS VOMS plugin: ready for integration -LCAS server implementation: ready mid-September -Job repository: ready end of September -The Fault Tolerance framework: we will have the internal integration in 2 weeks, so if everything goes fine, it will be ready mid-September. -The new version of the RMS (including support for Condor and LSF) is not ready , it still needs a few weeks of work, it will be ready at the end of September. -Not all the planned additions for the Fabric Monitoring framework (Oracle and MySQL server interfaces, GUI) will be ready, the only one (already agreed with Cal) is a simple web page to show the daemon status as read from the monitoring repository. It will be ready in 1-2 weeks. ------------------- WP 5 ------------------------------- From: "Gordon, JC (John) " 1. secure mode webservice. This is all ready and tested but needs agreement on configuring secure mode tomcat in case it breaks other people, mainly WP3. This will correctly identify users to the SE so that file ownership is correct. We are not planning to stop insecure mode until everyone else is happy. This is a prerequisite for (3) 2. Blocking SRM calls. We have implmented sufficient SRM calls to support GFAL. The calls are still blocking and use the existing SE in insecure mode. This is ready and is currently being tested. 3. GACLs - we have a GACL handler written that we can apply to existing functions to control fileaccess. We are now integrating it into some of the existing commands create and cache to start with which is about another weeks work.. This isn't full file security but is sufficient to demonstrate that we can deliver it. 4. Async SRM. Development of this is not yet complete. Parts have been tested but needs all to work. At least a couple of weeks work to deliver async version of what GFAL needs. 5. There are still outstanding bugs to be fixed but I don't think they feature separately on the release plan. ----------------------- WP 8 --------------------------------- From: frank harris Reply regarding WP8 priorities for 'post-EDG2.0' work ----------------------------------------------------------- F Harris 25/8/03 This reply follows comments from several loose cannons and 2 of the experiments (ATLAS and CMS), togeter with FH doing a reality check on what might be feasible in the limited time from early September to 25 September. We took note of our priorities as defined in the AWG document, but tempered this by a knowledge of what might be feasible in such a short period. We remind that the system is supposed to deal with thousands of jobs and millions of files etc (see AWG recommendations and talks at last review),and be highly stable... We need to also remind ourselves that the absolute priority is for supporting the released 2.0. Otherwise we risk to have nothing that resembles a stable 'quasi production quality' system. (I send a separate mail about WP8 view of 2.0 status as is now i.e. outstanding bugs, and features still to be tested) So, given the above overriding priority, the priorities are 1) Upgrade to gcc 3.2.2 as standard compiler (a very strong request from experiments) 2) Deploy VOMS (this is ready to go we believe) 3) Integrate VOMS with services (Data Management etc... 4) Full deployment of RLS One draws a line here in the sense that one believes we are already up to Sep 25! However we note that the following are also very important 5) Fault tolerance in RGMA registry 6) DAGMAN work. We note that that any work on 'space management' seems to have stopped. This will contine to cause problems and is a high priority item after the above list. (i.e. at 7 in the above list) ---------------------- LCG ------------------------------ From: "Zdenek Sekera" From: "Ian Bird" 1. Recompile all current LCG software for gcc3.2. This would create the "identical" version to compare with current LCG-1 (compiled on gcc2.95). 2. Add VOMS. 3. Add RLI. 4. Any other software/features will be included if we can show the resulting software is at least equal or better (in stability, performance) than the current one. As far as VDT/Globus go, I want us to move to VDT version 1.1.10 (at least) which is then the re-aligned US and LCG version (they currently have different releases for us). I would then like to push to Globus 2.4 by the end of the year. This is where the current middleware implementation should stay over the next year. Globus 3 is something that we should not consider before EGEE (although we might well deploy a development service based on GT3 as soon as possible next year). ------------------------------------------------------------------- WP7 does not intend to release new components no input received from WP9 and WP10