6533b82afe1ef96bd128c23a
RESEARCH PRODUCT
The IceProd framework: distributed data processing for the IceCube neutrino observatory
Joshua PepperE. MiddellElisa BernardiniJenni AdamsP. B. PriceS. HickfordR. G. StokstadP. BerghausJ. KläsK. RawlinsJ. FeintzeigK. FrantzenM. StamatikosS. EulerE. JacobiJ. A. GoodmanJ. KunnenT. O. B. SchmidtA. ChristovF. ClevermannM. DayA. BernhardJames MadsenT. R. WoodOscar MaciasM. N. TobinJ. C. GallagherM. LeuermannJ. P. YanezKurt WoschnaggPaul EvensonR. StrömM. KrasbergA. O'murchadhaTeresa MontaruliDamian PielothA. M. BrownA. TepeH.-p. BretzFrank McnallyDonglian XuS. De RidderM. LabareAllan HallgrenJavier GonzalezS. HussainD. T. GrandmontGiorgio MaggiO. SchulzT. FuchsD. J. KoskinenSandro KopperM. CasierG. W. SullivanD. Z. BessonA. GoldschmidtP. GretskovJ. P. RodriguesK. KringsC. De ClercqL. KöpkeDmitry ChirkinAlexander KappesAnatoli FedynitchA. Van OverloopN. A. StanishaCarsten RottD. HeeremanJ. LünemannE. A. StrahlerL. SchulteB. EichmannY. SestayoM. SchmitzK. JeroMatthias WolfS. BöserNathan WhitehornT. StezelbergerS. M. SabaR. HellauerB. RuzybayevR. C. BayM. RibordyS. CoendersA. Haj IsmailBenedikt RiedelDavid A. WilliamsMarkus AckermannS. W. BarwickA. StasikMaria TselengidouS. SchönebergM. Lesiak-bzdakP. RedlT. WaldenmaierTimo KargJ. Van SantenRezo ShanidzeJuan Carlos Diaz-velezK.-h. BeckerMatt DunkmanT. Fischer-waselsS. SchoenenThorsten GlusenkampCarlos ArguellesK. D. De VriesK. ClarkL. RädelM. KowalskiK. SchattoG. KohnenA. SchukraftHermann KolanoskiM. BissokDennis SoldinR. NahnhauerTodor StanevC. WendtC. KopperG. GolupT. FeuselsM. UsnerS. YoshidaH. TaavolaM. DanningerL. MohrmannK. HoshinaKirill FilimonovJ. E. JacobsenSubir SarkarSubir SarkarM. QuinnanMarcos SantanderE. BlaufussC. PfendnerN. MilkeGeorge JaparidzeDirk RyckboschR. BruijnG. MerinoW. HuelsnitzA. TamburroGlenn SpiczakD. J. BoersmaJ. C. DavisSegev BenzviK. JagielskiR. HoffmannL. GerhardtL. GerhardtI. RaeJ. C. GrohDavid SchultzGary BinderGary BinderSpencer KleinSpencer KleinMatthias VraegheXianwu XuC. WalckAxel GroßR. EaganU. NaumannMohamed RameezM. VehringD. F. CowenP. O. HulthJ. EischH. LandsmanKael HansonC. HaC. HaKara HoffmanA. H. Cruz SilvaA. KriestenSurujhdeo SeunarineJ. MillerC. BohmThomas K. GaisserG. B. YodhElisa ResconiK. HelbingThomas MeuresCh. WeaverO. JlelatiJ. KirylukJuanan AguilarD. R. NygrenTakao KuwabaraWolfgang RhodeS. TilavJ. BlumenthalJ. J. BeattyJ.-h. KöhneChad FinleyJ. ZiemannXinhua BaiB. ChristyJ. Becker TjusM. MerckH. WissingA. HomeierL. ClassenHans NiederhausenPaolo DesiatiA. IshiharaJ. PosseltJ. AuffenbergNaoko KurahashiN. Van EijndhovenChristopher WiebuschTim RuheDaniel BindigJ. L. KelleyMarkus AhlersOlga BotnerA. StößlA. ObertackeB. KaminskyS. OdrowskiJ. DaughheteeM. W. E. SmithReina H. MaruyamaK. MaseV. BaumM. J. LarsonG. KrollA. R. FazelyMike RichmanM. WallraffM. VogeA. FranckowiakA. OlivasSamvel Ter-antonyanL. BrayeurRasha AbbasiF. HuangM. G. AartsenJ. LeuteD. BerleyG. C. HillGerald PrzybylskiRené ReimannE. PinatM. ZollMichael J. BakerJ. CaseyLarissa PaulF. ScheriauL. GladstoneS. CohenDirk HeinenD. SeckelS. MiareckiS. MiareckiI. TaboadaDustin HebeckerTyce DeyoungP. A. ToaleS. FlisSimona ToscanoAlbrecht KarleC. Pérez De Los HerosA. SchönwaldE. UngerD. AltmannK. MeagherK. WiebeS. ZierkeA. OmairatP. HallenK. HultqvistDarren GrantSarah NowickiM. KauerD. BoseB. EberhardtH. S. MatisFrancis HalzenP. ZarzhitskyO. FadiranChristian SpieringC. SheremataS. WesterhoffG. TešićR. MorseSofia VallecorsaN. L. StrotjohannH. G. SanderM. WellonsDariusz Gorasubject
FOS: Computer and information sciencesMonitoringComputer scienceComputer Networks and CommunicationsDistributed computingData managementReal-time computingDistributed managementcomputer.software_genre01 natural sciencesData managementIceCube Neutrino ObservatoryTheoretical Computer ScienceIceCubeArtificial Intelligence0103 physical sciences010306 general physicsData processingData management; Distributed computing; Grid computing; Monitoring010308 nuclear & particles physicsbusiness.industryDistributed computingGrid computingComputer Science - Distributed Parallel and Cluster ComputingHardware and ArchitectureMiddleware (distributed applications)MiddlewareGrid computingParticleDistributed Parallel and Cluster Computing (cs.DC)Neutrinoddc:004businesscomputerSoftwaredescription
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. (C) 2014 Elsevier Inc. All rights reserved.
| year | journal | country | edition | language |
|---|---|---|---|---|
| 2015-01-01 |