Minutes of storage phone conference, 2005-11-16 Present: Durham: Mark Edinburgh: Greig, Phil Glasgow: Jamie Imperial: Olivier, Mona Lancaster: Matt Manchester: Alessandra QMUL: Giuseppe RAL Tier 1: Derek RAL Storage: Jens (chair+mins), Owen, Chris, Jiri Anyone else who joined after the roll call, or didn't say anything, let me know. Apologies: Glasgow: Graeme. Graeme sent an update (and other goodies) by email. 0. Quick action review, see below. 1. Quick site delta. Going up, Going down, Lying over (theorems about mapping prime ideals in commutative ring theory, or points in algebraic geometry. :-P ) The status and plans of the following sites are still unknown: * Sheffield Greig and Alessandra have been in touch; are believed to have tried installing dCache but status is unknown. Need to migrate their Classic SE. ACTION Jens: why should NorthGrid run dCache only? Group is not overly concerned yet, but need commitment. No sheffieldian is yet on the mailing list. ACTION Jens: poke (very mild and friendly escalation :-) * UCL UCL splits into Physics and CCC. Olivier and Greig will work with Physics on DPM install this week. CCC status unknown but expected to install DPM. Contact is Alice Fage who is not yet on the mailing list. We also need to cover a few was-there or nearly-there sites: * Liverpool Michael on leave. Paul wants dCache 1.6.6. ACTION Owen: clarify the need for 1.6.6, with Alessandra in the loop. * QMUL QMUL is a happy bunny. When QMUL is happy, we're happy. * LeSC LeSC publishes the IC HEP SE (dCache) but will be installing DPM as soon as David has time. I didn't expect that we need to cover Lancaster, and we didn't. 2. dCache 1.6.6. Owen has got 1.6.6 running. YAIM for 1.6.6 is in progress (Owen), can now set up the database. Configuration is very different (simpler!) in 1.6.6 than in < 1.6.6. Greig is testing the 1.5.X -> 1.6.6 upgrade; so far has managed to migrate the database (gdm to postgres). Watch the Wiki under dCache for news. Tier 1 is running 1.6.5 (1.6.5.3 I believe) and plan to test the upgrade at some point (Derek). Some FTS problems, particularly to Glasgow, see mailing list. Jamie reports 100 MB transfers work, but slow, 4-5 mins per file. And 1 GB files transfers falling over. ACTION Jamie: get performance numbers (Graeme has some?) and publish to list. For the problems, follow the track on the list. Note: RAL has FTS version 1.3.0 (IIRC). 3. VO space usage publishing via GIP Graeme reports that he has agreed with Jean-Philippe Baud and James Casey that he can do some work with GIP. So for all of you doing GIP stuff, keep in touch with Graeme (or watch the Wiki). Owen is working on the GIP side for dCache 1.6.6; using work from the previously reported Italian student. Don't expect much yet; Owen is only 1 person and is still hacking YAIM. 4. Storage goals for SC4 (Jens) Jens explained the rationale behind the current storage goals for SC4, essentially that because 2.1s may have short-to-medium-term interoperability problems at the SRM protocol level, we should set goals in terms of FTS which will be able to talk to all implementations. Some people voiced concerns about the timeline: e.g., if the dCache SRM 2.1 is delayed, it will affect some of our goals. Graeme has reported that phedex can call FTS, so CMS won't be left out. 5. AOB It was suggested that this list/group also look into data management. However, we have 41 (distinct) members, and not all would perhaps appreciate a wider focus. It was agreed that it is within the scope of this list/group to discuss also FTS and related documentation, but no further issues. ------------------------------------------------------------------------ ACTIONS * OK, I'm going to shake up the action list a bit. * Many are dup'd as bugs in Savannah. * I have located links to (some) documentation and added it to the bug. If I have thought it appropriate I closed the bug and the corresponding action. * Can I remind people to USE THE BUG TRACKER. That's what it's for. * Before you close a bug, please add a link to web docs: the wiki, or the dCache book, or Owen's dCache howto (bear in mind that Owen's howto is autogenerated so deep links move). 8 09/02/2005 Resurrect SRM client API Jiri Open Low priority This is now moving ahead; Jiri has started looking into it. I know how to do it in Java (including WSDL conversion, but minus the security side), so will add a recipe to the wiki. 31 15/06/2005 BDII howto required. Owen Open Is "How do I publish information about my SRM?" in the dCache FAQ sufficient? If not, go to the bug https://savannah.cern.ch/bugs/?func=detailitem&item_id=10106 and reopen it and solve it. And then close it. => Closed. 33 13/07/2005 Add HOWTO to GridPP web site Owen Open Evidently no longer needed. => Closed. 35 13/07/2005 Find out correct behaviour for what to do on full system Owen Ongoing Currently testing on 1.6.6. Owen requested clarification. Procedure for sysadmin _is_ documented. https://savannah.cern.ch/bugs/index.php?func=detailitem&item_id=10113 Unfortunately, the bug was closed without any information about the solution!! I added a link to an entry in the dCache FAQ created by Greig. http://wiki.gridpp.ac.uk/wiki/DCache_FAQ#What_happens_if_a_pools_local_filesystem_fills_up.3F Not yet tested on 1.6.6. Don't know if it's tested on 1.6.5. The other aspect of this is whether volatile files are removed as the system fills up. 41 10/08/2005 Agree licence with DESY Jens Open Last poked Mon 14 Nov 2005. No news. With external lawyers for comments. 50 14/09/2005 Can DPM use space on WNs (Durham) Owen Open Low priority No news. 51 14/09/2005 Explain CE publishing close classic and not close SRM Owen Open In Edinburgh, the DefaultSE is the DPM, for RAL the DefaultSE is dCache. So it is possible. => Closed. 53 12/10/2005 Find reasoanable % for SE uptime for SC4 Jeremy Open Reassigned to Jeremy to follow up with higher level mgmt. 54 02/11/2005 Report on performance/scalability with pools on WNs Paul Open No news. Would presumably solve #50 as well. 55 02/11/2005 Report on upgrade to 1.6.6 Greig Open Reassigned Reassigned to Greig now. See item on agenda. So far he has upgraded the database. Greig's summary should appear on http://wiki.gridpp.ac.uk/wiki/Ed_Upgrade_152_To_166 56 02/11/2005 Investigate glite install and report Graeme Open Need to clarify priority A mail was circulated to the list last week. ** PLEASE DO NOT REDISTRIBUTE ** (request from Fraser). => Closed. 57 02/11/2005 Investigate StoRM UNASSIGNED Open Wait for INFN report Need to find out who is doing the testing. 58 09/11/2005 Circulate EGEE StoRM presentations to list Jens Open Arrr. 'Tis done. Arr. 59 09/11/2005 Follow up with Olivier & David McBride to get LeSC signed up Jens Open 'Tis done. Welcome David. ------------------------------------------------------------------------ NEW ACTIONS 60 16/11/2005 Add client wsdl->library recipe to wiki Jens Open 61 16/11/2005 Figure out who in INFN is testing StoRM Jens Open 62 16/11/2005 Follow up with JC about NorthGrid being dCache only Jens Open High priority 63 16/11/2005 Follow up with Sheffield about timeline Jens Open 64 16/11/2005 Publish FTS performance to list Jamie Open When I find a good way of counting the bugs, I'll do that regularly. Note that many bugs (all but 8 out of 32) are unassigned. Obviously bugs are not a good progress metric :-(