Agenda

The agenda below is subject to change. Please check back regularly for any updates. Starter questions are provided as suggestions by the organizing committee to help guide each discussion session topics.

Technical Workshop - Day 1

November 8, 2021 from 15:00 to 19:00 UTC

Session 1 (15:00-16:00 UTC)

  • (5 min) Introductory Remarks
  • (15 min) Presentation - API design of the Rubin alert distribution system (Spencer Nelson)
  • (45 min) Discussion - Data Transport Mechanisms (Kafka, Avro, PubSub, etc.)
    • What technologies are people using (e.g., Kafka, Avro, PubSub)?
    • What are the benefits and challenges to each of these?
  • (15 min) Break

Session 2 (16:15-17:15 UTC)

  • (15 min) Presentation - Setting up "SSL" for Kafka (Adam Scott)
  • (45 min) Discussion - Connection Between PPDB and Brokers
    • Will LSST support syncing the schema / data so that Broker DBs correctly/authoritatively mirror PPDB products?
    • Do we keep separate copies for data rights holders and public users?
    • Who pays for hosting duplicate PPDB?
  • (15 min) Break

Session 3 (17:30-18:30 UTC)

  • (15 min) Presentation - Pub/Sub and Kafka (Troy Raen)
  • (45 min) Discussion - Supporting user data access: streaming V.S. database access
    • What are the data access patterns that we care about (i.e., what user stories are we developing around)?
    • Are we focusing on streams too much?
    • How does this inform our data storage solution?
    • What are the database technologies being used?
    • How do data storage solutions differ in development time, maintenance, cost, and scalability?

Technical Workshop - Day 2

November 9, 2021 from 15:00 to 19:00 UTC

Session 4 (15:00-16:00 UTC)

  • (15 min) Presentation - Distributed database for time-series using Apache HBase: lessons learned (Julien Peloton)
  • (45 min) Discussion - Standardization and interoperability between brokers
    • Is there a clear idea on what should be standardized?
    • Are there standards we should adopt?
    • Are there key design questions, or are first-mover decisions largely fine?
    • Are there common code-bases we could be sharing/developing?
  • (15 min) Break

Session 5 (16:15-17:15 UTC)

  • (15 min) Presentation - Real time host galaxy identification from multi resolution images (Francisco Forster)
  • (45 min) Discussion - Supporting reproducible analyses in the long term
    • How do we guarantee an analysis can be rerun 1, 5, 10, or more years from now?
    • Reproduction includes services, not just data
  • (15 min) Break

Session 6 (17:30-18:30 UTC)

  • (15 min) Presentation - Organizing spatially indexed data with Cassandra (Ken Smith)
  • (45 min) Discussion - Cross broker services and communication
    • How to handle cross matching across brokers?
    • What non-rubin data sources are we considering?
    • Who determines/maintains the data rights permissions for each user / group?
    • What mechanism should be used for passing authentication information around?

Technical Workshop - Day 3

November 10, 2021 from 15:00 to 19:00 UTC

Session 7 (15:00-16:20 UTC)

  • (10 min) Hack day Kick off
  • (60 min) Hack sessions in breakout rooms
  • (10 min) Checkpoint (What have you accomplished / are stuck on)
  • (60 min) Break

Session 8 (17:20-18:40 UTC)

  • (60 min) Resume breakout rooms
  • (20 min) Presenting accomplishments and future plans.