相关论文: Petabyte Scale Data Mining: Dream or Reality?
Sky surveys represent a fundamental data basis for astronomy. We use them to map in a systematic way the universe and its constituents, and to discover new types of objects or phenomena. We review the subject, with an emphasis on the…
Astronomy is one of the most data-intensive of the sciences. Data technology is accelerating the quality and effectiveness of its research, and the rate of astronomical discovery is higher than ever. As a result, many view astronomy as…
The massive data sets from today's particle physics experiments present a variety of challenges amenable to the tools developed by the statistics community. From the real-time decision of what subset of data to record on permanent storage,…
The Sloan Digital Sky Survey (SDSS) is a project to definitively map $\pi$ steradians of the local Universe. An array of CCD detectors used in drift-scan mode will digitally image the sky in five passbands to a limiting magnitude of $r'…
As the volume of data grows, astronomers are increasingly faced with choices on what data to keep -- and what to throw away. Recent work evaluating the JPEG2000 (ISO/IEC 15444) standards as a future data format standard in astronomy has…
The domain of radio astronomy is currently facing significant computational challenges, foremost amongst which are those posed by the development of the world's largest radio telescope, the Square Kilometre Array (SKA). Preliminary…
Astrophysics and Space Science are becoming increasingly characterised by what is now known as "big data", the bottlenecks for progress partly shifting from data acquisition to "data mining". Truth is that the amount and rate of data…
Observational astronomy has changed drastically in the last decade: manually driven target-by-target instruments have been replaced by fully automated robotic telescopes. Data acquisition methods have advanced to the point that terabytes of…
Modern geosciences have to deal with large quantities and a wide variety of data, including 2-D, 3-D and 4-D seismic surveys, well logs generated by sensors, detailed lithological records, satellite images and meteorological records. These…
Recently, we have been witnessing huge advancements in the scale of data we routinely generate and collect in pretty much everything we do, as well as our ability to exploit modern technologies to process, analyze and understand this data.…
In this paper we have focused a variety of techniques, approaches and different areas of the research which are helpful and marked as the important field of data mining Technologies. As we are aware that many Multinational companies and…
Ongoing observations of the Cosmic Microwave Background, such as the MAXIMA and BOOMERanG projects, are providing datasets of unprecedented quality and ever-increasing size. Exact analysis of the data they produce is a serious computational…
The 3.2 giga-pixel LSST camera will produce approximately half a petabyte of archive images every month. These data need to be reduced in under a minute to produce real-time transient alerts, and then added to the cumulative catalog for…
LSST promises to be the largest optical imaging survey of the sky. If we were fortunate enough to have the equivalent of LSST today, it would represent a "fire hose" of data that would be difficult to store, transfer, and analyze with…
In time-domain astronomy, we need to use the relational database to manage star catalog data. With the development of sky survey technology, the size of star catalog data is larger, and the speed of data generation is faster. So, in this…
Unique scientific instruments designed and operated by large global collaborations are expected to produce Exabyte-scale data volumes per year by 2030. These collaborations depend on globally distributed storage and compute to turn raw data…
Despite the large budgets spent annually on astronomical research equipment such as telescopes, instruments and supercomputers, the general trend is to analyse and view the resulting datasets using small, two-dimensional displays. We report…
The BaBar database has pioneered the use of a commercial ODBMS within the HEP community. The unique object-oriented architecture of Objectivity/DB has made it possible to manage over 700 terabytes of production data generated since May'99,…
Policy Brief on "Global Data in Astronomy: Challenges and Opportunities", distilled from the corresponding panel that was part of the discussions during S20 Policy Webinar on Astroinformatics for Sustainable Development held on 6-7 July…
Astronomy is entering a new era as multiple, large area, digital sky surveys are in production. The resulting datasets are truly remarkable in their own right; however, a revolutionary step arises in the aggregation of complimentary…