Please, see the linked thread on the community for official communication.

Feel free to rant and vent in this lemmy-thread, but don’t speculate or “give good advice” to the operations-team; they know what they are doing.

  • Eevoltic@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 days ago

    Updates: It should be operational as of tonight. OSM are switching to their new ISP.

    https://community.openstreetmap.org/t/openstreetmap-org-currently-offline-operations-team-are-working-to-restore-15-december-2024-updated/122930/53

    You heard it here first…

    We should have all service back by the European evening tomorrow (Tuesday). We are switching to the new ISP.

    (Minor matter of me having to get new ISP setup working on our routers with some virtual networking required for simultaneous upstream ISPs, and some special VRRP setup required for the new ISP redundancy setup)

  • Possibly linux@lemmy.zip
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    2 days ago

    TL;DR

    The OSM team is very small and constrained heavily by financials. They had a single ISP which acted as a single point of failure. They are moving to a new ISP but they expect the outage to last until Wednesday

  • fmstrat@lemmy.nowsci.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    3 days ago

    We have dual redundant links via separate physical hardware from our side to our Tier 1 ISP. We unexpectedly discovered their equipment is a single point failure.

    Our Tier 1 ISP has just confirmed hardware failure of the upstream router (after 16 hours!) and will provision new replacement device once it has been shipped to site.

    I wonder why the OSM team isn’t calling the ISP out by name. Not having on-site spares at a Tier 1 is just, insane.

    • pietervdvn@lemmy.mlOPM
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 day ago

      I wonder why the OSM team isn’t calling the ISP out by name.

      Because this would open up OSM for defamation (and hinders a commercial resolve). It doesn’t solve anything right now.

    • GreatAlbatross@feddit.uk
      link
      fedilink
      English
      arrow-up
      0
      ·
      3 days ago

      Amazingly, this isn’t the first time I’ve heard of this happening.

      Charging for a redundant line, then discovering that it actually wasn’t one day when it goes down.

      • fmstrat@lemmy.nowsci.com
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 days ago

        “We paid for a redundant line but it wasn’t redundant!”

        ‘Sure it was. They just went to the same router.’

        Can you imagine the straight face that tech must have had to keep?