Keyoxide: https://keyoxide.org/9f193ae8aa25647ffc3146b5416f303b43c20ac3

OpenPGP: openpgp4fpr:9f193ae8aa25647ffc3146b5416f303b43c20ac3

  • 4 Posts
  • 13 Comments
Joined vor 3 Monaten
cake
Cake day: Nov. 08, 2022

help-circle
rss

Wow the rendering is much better/faster now 🌠


I don’t know about language models in specific. I read this recently on “federated learning” https://venturebeat.com/ai/federated-learning-key-to-securing-ai/

It says data privacy issues. Maybe it is also a more complex architecture.


cross-posted from: https://group.lt/post/44860 > Developers across government and industry should commit to using memory safe languages for new products and tools, and identify the most critical libraries and packages to shift to memory safe languages, according to a study from Consumer Reports. > >The US nonprofit, which is known for testing consumer products, **asked what steps can be taken to help usher in "memory safe" languages, like Rust**, over options such as C and C++. Consumer Reports said it wanted to address "industry-wide threats that cannot be solved through user behavior or even consumer choice" and it identified "memory unsafety" as one such issue.  > >The [report](https://advocacy.consumerreports.org/research/report-future-of-memory-safety/), Future of Memory Safety, looks at range of issues, including challenges in building memory safe language adoption within universities, levels of distrust for memory safe languages, introducing memory safe languages to code bases written in other languages, and also incentives and public accountability. More information: - https://advocacy.consumerreports.org/research/report-future-of-memory-safety/ - https://advocacy.consumerreports.org/wp-content/uploads/2023/01/Memory-Safety-Convening-Report.pdf
fedilink

The hardest scaling issue by Codeberg (a nonprofit, free software platform/service for code hosting)
cross-posted from c/softwareengineering@group.lt: https://group.lt/post/44632 > This kind of scaling issue is new to Codeberg (a nonprofit free software project), but not to the world. All projects on earth likely went through this at a certain point or will experience it in the future. > > When people like me talk about scaling... It's about increasing computing power, distributed storage, replicated databases and so on. There are all kinds of technology available to solve scaling issues. So why, damn, is Codeberg still having performance issues from time to time? > > ...we face the "worst" kind of scaling issue in my perception. That is, if you don't see it coming (e.g. because the software gets slower day by day, or because you see how the storage pool fill up). Instead, it appears out of the blue. > > **The hardest scaling issue is: scaling human power.** > > Configuration, Investigation, Maintenance, User Support, Communication – all require some effort, and it's not easy to automate. In many cases, automation would consume even more human resources to set up than we have. > > There are no paid night shifts, not even payment at all. Still, people have become used to the always-available guarantees, and demand the same from us: Occasional slowness in the evening of the CET timezone? Unbearable! > >I do understand the demand. We definitely aim for a better service than we sometimes provide. However, sometimes, the frustration of angry social-media-guys carries me away... > > two primary blockers that prevent scaling human resources. The first one is: trust. Because we can't yet afford hiring employees that work on tasks for a defined amount of time, work naturally has to be distributed over many volunteers with limited time commitment... second problem is a in part technical. Unlike major players, which have nearly unlimited resources available to meet high demand, scaling Codeberg's systems... TLDR: sustainability issues for scaling because Codeberg is a nonprofit with much limited resources, mainly human resources, in face of high demand. Non-paid volunteers do all the work. So needs more people working as volunteers, and needs more money.
fedilink

You’re a person of culture as well I see; I upvote comments of culture yes📠

I remember talking with you at the NixOS matrix; nice to see you here as well💖✨✨✨🌠


It is because it departs from POSIX that it is good; I recognize the syntax for some functionality is cumbersome and hard to remember though. There are similarities like command names and piping still…

I use NixOS and home-manager, so for switching I just

  home-manager.users.yuu = {
    programs.nushell = {
      package = pkgs-update.nushell;
      enable = true;
      configFile.source = ../../config/nushell/config.nu;
      envFile.source = ../../config/nushell/env.nu;  
    };
  };

The config.nu and env.nu is basically the default just with a customized prompt.

Then in my alacritty.ylm I set shell to the nu binary

shell:
  program: /etc/profiles/per-user/yuu/bin/nu

Also learned from official resources https://www.nushell.sh/book. When I have doubts, I ask either on Nushell’s GitHub discussions or https://matrix.to/#/#nushell:matrix.org

And to keep a POSIX shell

{
  environment = {
    systemPackages = with pkgs; [
      mksh
    ];

    sessionVariables = rec {
      TERM = "alacritty";
      TERMINAL = "alacritty";
      SHELL = "${pkgs.mksh}/bin/mksh";
    };

  environment.shells = [
    "${pkgs.mksh}/bin/mksh"
  ];
}

You can use Nix which works in many distros; it has the most packages of any package repository/collection

https://nixos.org/download.html#nix-install-linux

GNU Guix is similar, but not as much packages

https://guix.gnu.org/en/download/ https://guix.gnu.org/manual/en/html_node/Binary-Installation.html


I use nushell for my terminal/console (alacritty). For POSIX compability, mksh; I set it as SHELL so programs, which expect/assume POSIX, use it instead of nu. This is the way to have best of both worlds.


do we need to create an acc for every site, or just one? would join if it federates between communities


For nixos /etc/nixos/flake.nix. Example https://git.sr.ht/~misterio/nix-config/tree/main/item/flake.nix

For home-manager see https://nix-community.github.io/home-manager/index.html#ch-nix-flakes

For individual projects like that Pytorch one you can put on any git repo.

If you use same nixpkgs revision as the one you currently have using channels nix should not rebuild derivations.


nix flake update

And to add a new flake to flake.nix

inputs.my-flake.url = "github:owner/repo";

✨✨✨✨✨✨✨✨✨✨✨✨

Maybe there is a way to add flakes through the command line which I do not know of.


https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html

or man nix3-flake.

For a NixOS flake example: https://git.sr.ht/~misterio/nix-config/tree/main/item/flake.nix

For specific language examples https://github.com/NixOS/templates (which you can nix flake new my-project-name --template "templates#template-name". For real examples https://sourcegraph.com/search?q=context:global+.*+file:flake.nix+lang:Nix&patternType=regexp&sm=1

here a pytorch example when I was learning Flakes

# https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html
# https://discourse.nixos.org/t/pytorch-cuda-on-wsl/18267
# https://discourse.nixos.org/t/pytorch-and-cuda-torch-not-compiled-with-cuda-enabled/11272
# https://gitlab.com/abstract-binary/nix-nar-rs/-/blob/main/flake.nix
# https://github.com/hasktorch/libtorch-nix
# https://github.com/google-research/dex-lang/blob/main/flake.nix
# https://yuanwang.ca/posts/getting-started-with-flakes.html

{
  description = "PyTorch";

  # Specifies other flakes that this flake depends on.
  inputs = {
    devshell.url = "github:numtide/devshell";
    utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:nixos/nixpkgs/nixos-22.11";
  };

  # Function that produces an attribute set.
  # Its function arguments are the flakes specified in inputs.
  # The self argument denotes this flake.
  outputs = inputs@{ self, nixpkgs, utils, ... }:
    (utils.lib.eachSystem [ "x86_64-linux" ] (system:
      let
        pkgs = (import nixpkgs {
          inherit system;
          config = {
            # For CUDA.
            allowUnfree = true;
            # Enables CUDA support in packages that support it.
            cudaSupport = true;
          };
        });
      in rec {
        # Executed by `nix build .#<name>`
        packages = utils.lib.flattenTree {
          hello = pkgs.hello;
        };

        # Executed by `nix build .`
        defaultPackage = packages.hello;
        # defaultPackage = pkgs.callPackage ./default.nix { };

        # Executed by `nix develop`
        devShell = with pkgs; mkShell {
          buildInputs = ([
            python39 # numba-0.54.1 not supported for interpreter python3.10
          ] ++ (with python39.pkgs; [
            inflect
            librosa
            pip
            pytorch-bin
            unidecode
          ]) ++ (with cudaPackages; [
            cudatoolkit
          ]));

          shellHook = ''
            export CUDA_PATH=${pkgs.cudatoolkit}
          '';
        };
      }
    ));
}

nix-channel works now and is a lot simpler

It is not. Once you understand flakes, you will see how much better it is. If you do not understand why flakes exist to begin with, read https://www.tweag.io/blog/2020-05-25-flakes/

also use in conjunction with flakes:

  • direnv, nix-direnv
  • devshell

Flakes are easier and better than channels. Use it instead.


NixOS 22.11 released 🎉🎉🎉
cross-posted from: https://group.lt/post/30446 > **1652 contributors**, who authored **30371 commits** since the previous release. > > NixOS is already known as the **most up to date distribution** while also being the distribution with the **most packages**. > > This release saw **16678 new packages** and **14680 updated packages** in nixpkgs. We also **removed 2812 packages** in an effort to keep the package set maintainable and secure. In addition to packages the NixOS distribution also features modules and tests that make it what it is. This release brought **91 new modules and removed 20**. In that process we **added 1322 options and removed 487**.
fedilink

NixOS 22.11 released 🎉🎉🎉
cross-posted from: https://group.lt/post/30446 > **1652 contributors**, who authored **30371 commits** since the previous release. > > NixOS is already known as the **most up to date distribution** while also being the distribution with the **most packages**. > > This release saw **16678 new packages** and **14680 updated packages** in nixpkgs. We also **removed 2812 packages** in an effort to keep the package set maintainable and secure. In addition to packages the NixOS distribution also features modules and tests that make it what it is. This release brought **91 new modules and removed 20**. In that process we **added 1322 options and removed 487**.
fedilink

Sparkles are the bestt✨✨✨✨