Minutes of storage phone conference 22 Feb 2006 Present: Glasgow: Graeme, Jamie Durham: Mark Lancaster: Matt RAL Tier 1: Derek, Andrew Liverpool: Paul RAL Storage: Jiri, Owen, Jens (chair+mins) 0. Review of actions (see below) 1. Hot off the press. Problems with FTS coexistence with srmcp? Is not a problem, probably a coincidence. 2. Post GGF report on storage (Jens) There was a question whether SRB and SRM storage should merge across the Grid in a Grid Interoperability Now workshop; but Jens thinks there are two problems with this approach: first SRBs are not SRMs - they do different things. Secondly, the user communities are disparate - more or less HEP and everybody else. The workshop agreed to keep two separate "islands". James Casey reported (private conversation) that DPM 1.5 is out, with security and access control integrated into the nameserver (so is respected by all access methods), and it uses GSI RFIO by default. It supports VOMS. People are keen to have an SE which does not need gridmap files, but DPM 1.5 still needs them to keep the Globus libs happy. On a related note, Graeme reports that apt-get-upgrading to 1.5.2 (with YAIM doing the config) does not bring the schema with it, so more work is needed. Graeme will investigate (ACTION). James also suggested (another conversation) that interfacing xrootd to the "slower" SRMs is not likely to make the experiments happy, because the result won't be xrootd as they know and love it, because xrootd aims to do different things. 3. dCache Matt (Lancaster) has volunteered. External testing is **urgently** needed because LCG is doing no further testing and Owen is keen to ensure that there are no RALisms in his YAIM code before it goes into LCG. Jens will speak to the CA manager (hat) to get the signer to speed up the process a bit. 4. Information provider Issues with GLUE storage has been raised with the LCG GDB. See Graeme's earlier eloquent emails: http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0602&L=gridpp-storage&T=0&O=D&P=1752 and http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0602&L=gridpp-storage&T=0&O=D&P=2215 It was discussed at CHEP as well, and out came the "BOF declaration". As a do for now thing, they want to change the semantics of the values so permanent now means "written to tape", and durable means "lives on disk forever". Graeme points out there are three dimensions: * Max access time/latency - the maximal time it will take the SE to get any file ready for transfer; * "Lifetime" in the SRM sense, volatile, durable, and permanent, as indicators of how the file is managed in the disk cache; * Quality/durability - how likely the file is to not vanish unintentionally. Of course an SE can offer more than one of the above. And for different VOs. So the usual scalability problems apply where you need to give the same SE more than one name (e.g. dcache-tape.blah and dcache.blah for disk) because the schema can't cope (depending on whether this stuff is published for the SE or the SA). The problem affects of course also SRM 1s who use the same GLUE schema, but LCG want to change the semantics for 2.1 only, so even if it's the *same* schema in the GRIS, the semantics is different between SRM 1s and 2s. It has been suggested that a 2.1 client can ask for specific values of the above when doing a prepareToPut, and the server should be able to offer which values the client can get. And the client can then decide whether to accept or not (cf. the way cache expiry time was supposed to be managed for durable and volatile files). Unless one does something Very Clever(tm) for 2.1, it will require an extension to SRM, effectively becoming WLCG SRM version 2.2. Outcome of the SC workshop, includes a mailing list managed by Maarten. Jens will post this summary to the wiki and add the link to, or text from, the BOF (ACTION). 5. 10 easy storage questions (Greig's idea...) We need to improve the discussion with the experiments a bit and Greig suggested 10 easy storage questions. So unlike the network ditto, this one is probably more for the *users* than the *sites*. We should create a page in the Wiki and add suggestions (ACTION Jens). Actually I have thought of creating an intentionally orphaned page and then parent it later. We don't really want to publish it till it's ready so maybe the Wiki isn't the best way. 6. SRM 2.1 tests The SRM 2.1 test client is now available from sourceforge http://s-2.sourceforge.net/ It comes with tests for each of the SRM 2.1 functions - see the result for DPM 1.4 https://wiki.gridpp.ac.uk/wiki/DPM_SRMv2_Status. If you start writing non-trivial tests (for higher level stuff), let us know. Jens thought he'd created a page in the Wiki for 2.1 tests but can't find it now - sigh - ACTION. 7. Site hardware recommendations Andrew Sansum is collecting experiences from the Tier 2s for the PMB, both from the cost perspective and the performance perspective. Jens reported that someone else is doing something (which is why it's on the agenda in the first place) but didn't rememeber who. [I think it was something similar that I happened to stumble across but now can't find again, an LCG attempt to define hardware guidelines, and thought oooh that's interesting because we've discussed this on and off for a while -- -j]. 8. Other post CHEP items. Skipped those since Greig is away. Greig and Graeme together sent a summary; the big issue is really the GLUE stuff we have discussed above. 9. xrootd for dCache? Greig reports there's a door. Do we need it, is anyone using it? We'll need follow up with the experiments. Maybe that should be one of the ten questions. Graeme reports that there is interest in rootd (no x) as an alternative transport protocol for DPM. And that J.-P. says it will not be too difficult. 10. AOB Owen asked about testing his new dCache deployment (and the information publishing in particular) (see also item 2). Derek and Graeme reported that FTS doesn't use the information system; Graeme and Jens reported that lcg-* and GFAL do, and, indeed, the FTS eventually will (or is supposed to), to recognise different SRMs from quite a long way away. Owen raised the issue of realistic testing without disrupting other people's stuff, and Graeme suggested hooking his SEs into the testzone BDII at Imperial. ************ ACTIONS ************ 41 10/08/2005 Agree licence with DESY Jens Open No news. Still on *my* [Jens'] todo list. 50 14/09/2005 Can DPM use space on WNs (Durham) Graeme Closed Low priority I've now closed this one because the result was published to the list (see minutes of 8 Feb). Instead, I opened a new action #89. 53 12/10/2005 Find reasoanable % for SE uptime for SC4 Jeremy Open Reassigned. Follow up with GDB et al No news. 54 02/11/2005 Report on performance/scalability with pools on WNs Paul Progress Paul will add stuff to the Wiki No news. 57 02/11/2005 Investigate StoRM Jiri Open Wait for INFN report 60 16/11/2005 Add client wsdl->library recipe to wiki Jens Open 61 16/11/2005 Figure out who in INFN is testing StoRM Jens Open Low priority DONE, at GGF. Vincenzo Vagnoni 78 25/01/2006 Get sites to forward details of their disk setups Greig Open No news. 82 01/02/2006 Produce file QoS summary document Jens Open Progress but still open. Becoming higher priority. See also discussion (item 4). 84 08/02/2006 Send email to chepers about SRM issues for preCHEP ws. Jens Open Done. 85 08/02/2006 Circulate agenda for pre-CHEP workshop Graeme Open Done. http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0602&L=gridpp-storage&T=0&O=D&P=2215 86 08/02/2006 Extend monitoring to do sites per VO and VOs per site Greig? Open 87 08/02/2006 Publish 2.1 testing tool by 10 Feb 2006 Jiri Open Done. http://s-2.sourceforge.net/ 88 08/02/2006 Report whether FTS 1.5 supports SRM 2.1 Graeme Open Graeme says no. CERN is using 1.4(.X) and Tier 1s 1.3(.Y); 89 21/02/2006 Clarify need for running DPM on WNs Jens Open Mark says "forget it." ************ NEW ACTIONS ************ 90 22/02/2006 Investigate DPM upgrade to 1.5 schema issue Graeme Open 91 22/02/2006 Add an SRM 2.1 tests page to the wiki Jiri Open See also #75 I have generously reassigned this to Jiri. 92 22/02/2006 Add a GLUE storage problem page to the wiki Jens Open 93 22/02/2006 Add 10 questions page to the Wiki? Jens Open