Radar Trends to Watch: May 2024 – O’Reilly

In the last month we’ve seen a blizzard of new language models. It’s almost hard to take this news into consideration, but Microsoft’s public (but possibly not open source) Phi-3 is definitely worth a look. We have also seen promising work in reducing the resources required to perform inference. This may lead to larger models, but it will also lead to reduced power usage for small and medium-sized models.

AI

• Microsoft’s Phi-3-mini is another freely available language model. It’s small enough to run locally on phones and laptops. Performance is similar to GPT-3.5 and Mixtral 8x7B.
• Google’s Infini-attention is a new inference technology that allows large-scale language models to provide infinite context.
• Companies are increasingly adding AI bots as observers to their boards of directors. Bots exist to plan strategy, help with financial analysis, and report on compliance.
• OutSystems provides a low-code toolkit for building AI agents named AI Agent Builder.
• Ethan Mollick’s Prompt Library is worth checking out. Most of his messages are collected from his books and blogs. Most are Creative Commons and require only attribution. Anthropic has also published a library of prompts for use with Claude, but it would probably work for other LLMs as well.
• There are many solutions for people who want to run large language models locally. These range from desktop apps to APIs. Here is the list:
• Meta has released 8B and 70B versions of Llama 3. The largest version has not yet been released. Initial reports suggest that these smaller versions imposing.
• Mistral AI has announced the Mixtral 8x22B, a larger version of its very impressive Mixtral 8x7B expert mixing model.
• Effort is a new way to perform LLM inference that reduces the amount of floating point computation required without compromising the results. Efforts have been implemented for Mistral, but should work on other models as well.
• ML Commons is developing AI safety benchmarks to test AI chatbots against common kinds of abuse. They warn that the current version (0.5) is only a proof of concept and should not be used to test production systems.
• The flagship Fine Tuning is a new technique for fine-tuning language models. This is unique because it focuses specifically on what you want your model to do. Not only is it faster and more efficient, but it also outperforms other fine-tuning techniques.
• AI systems can be more persuasive than humans, especially if they have access to information about the person they are trying to persuade. This extreme form of microtargeting could mean that AI has discovered persuasive techniques that we don’t yet understand.
• There were three major language model releases in 24 hours: Gemini Pro 1.5, GPT-4 Turbo, and Mixtral 8x22B. Mixtral is the most interesting. This is the larger successor to the Mixtral 8x7B, a very impressive professional mix model.
• More models for music creation are popping up everywhere. In addition to Stable Audio and Google’s MusicLM, there are Sonauto (which appears to be unrelated to Suno; Sonauto uses a different kind of model) and Udio.
• Ethical Applications to Deepfakes? Domestic data streamers create synthetic images based on memories, including important events that were not captured in photos. Interestingly, older image models seem to produce more satisfactory results than newer models.
• What happened after AlphaGo beat the world’s best Go player? Human Go players have improved. Some of the improvements came from studying the games the AI ​​plays. Some of that comes from increased creativity.
• You should listen to Permission Is Hereby Granted, where Suno sets the MIT license for music as a piano ballad.
• How does AI-based code completion work? GitHub doesn’t say much, but Sourcegraph has provided some details about the Cody helper. And since Cody is open source, you can analyze its code.
• Claude-llm-trainer is a Google Colab notebook that simplifies the learning process of Meta’s Llama 2.
• In a series of experiments, large-scale language models have proven to be better than “classical” models at financial time series forecasting.
• An easier way to run language models locally: the Opera browser now includes support for 150 language models. This feature is currently only available in developer streams.
• JRsdr is an AI product that automates all corporate social media. Do you dare believe it?
• LLMLingua-2 is a special model designed to compress prompts. Compression is useful for long prompts such as RAGs, chains of thought, and other techniques. Compression requires less context, ultimately improving performance and reducing costs.
• OpenAI shared some samples generated from its Voice Engine, a (yet-unreleased) model for synthesizing human voices.
• What generative AI can’t do: create plain white images. Perhaps it’s no surprise that it’s difficult.
• DeepMind has developed a large-scale language model to verify the accuracy of LLM output. Search-Augmented Factuality Evaluators (SAFEs) appear to be more accurate and less expensive to operate than crowdsourced humans. The SAFE code is posted on GitHub.
• AI-generated watermarks are often seen as a way to identify AI-generated text (and are required by law in the EU), but it is relatively easy to spot and remove watermarks or copy them for use on other documents.
• Small is not necessarily a disadvantage, especially when it comes to vision models. Smaller models trained on data that are carefully curated to be relevant to the task at hand are less prone to overfitting and other errors.

programming

• Martin Odersky, the creator of the Scala programming language, proposed “Lean Scala”, a simpler and easier to understand way to write Scala. Lean Scala is neither a new language nor a subset. This is the programming style of Scala 3.
• sotrace is a new tool for Linux developers that shows all the libraries a program is connected to. This is a great way to discover any supply chain dependencies. Try it; You’d be surprised, especially if you run it against a process ID rather than a binary executable.
• Aider is a nice little tool that facilitates pair programming with GPT 3.5 or 4. You can edit files in a Git repository by committing the changes with a generated descriptive message.
• Another new programming language: Vala. It is object-oriented, Java-like, compiled into native binaries, and can link to many C libraries.
• Great advice from Anil Dash: Create better documents. And in a similar vein, Write Readable Code by Gregor Hohpe.
• According to Google, programmers working in Rust are just as efficient as programmers working in Go, and twice as efficient as programmers working in C++.
• Winglang is a programming language for DevOps. This represents a higher level of abstraction for deploying and managing applications in the cloud. It includes a complete toolchain for developers.
• Tracking time has always been one of the most frustratingly complex parts of programming. Especially when you consider time zones. Now the moon needs its own time zone. Because for relativistic reasons, time passes slightly faster on the Moon.
• The Linux Foundation has started the Valkey project, a fork of the Redis database under an open source license. Redis is a popular in-memory key-value database. Like Terraform and other products, it has recently been relicensed under terms that are unacceptable to the source community.
• Redict is another fork of Redis, this time under LGPL. It is different from Valkey, which is a fork released by the Linux Foundation. Redict will focus on “stability and long-term maintenance” rather than innovation and new features.
• “Ship it” culture is destructive. Take the time to learn, understand, and document. It will pay off.

security

• GitHub allows comments to specify files that are automatically uploaded to the repository using automatically generated URLs. This feature is useful for bug reporting, but has been used by threat actors to inject malicious code into repositories.
• GPT-4 can read security advisories (CVE) and exploit vulnerabilities. Researchers haven’t been able to test the Claude 3 and Gemini yet, but other models don’t appear to have this feature.
• LastPass password manager users have been targeted by a relatively sophisticated phishing attack. The attack originated from the CryptoChameleon phishing toolkit.
• Protobom is an open source tool that makes it easier for organizations to create and use software bills of materials. Protobom was developed by OpenSSF, CISA, and DHS.
• The failed attack on xz Utils last month was probably not an isolated incident. The OpenJS Foundation also reported a similar incident, but did not specify which projects were affected.
• System Package Data Exchange (formerly known as Software Package Data Exchange 3.0) is a standard for tracking all supply chain dependencies, not just software. GitHub is integrating support for generating SPDX data from dependency graphs.
• The malicious PowerShell script used in many attacks is believed to have been created by AI. (That said, the script has comments for every line of code.) There will be more…
• Kobold Letters is a new email vulnerability and it’s a real headache. After an HTML-formatted email is delivered, a hostile agent can use CSS to modify the context in which the email is displayed.
• AI can hallucinate package names when generating code. These non-existent names often appear in software. Therefore, it is possible to observe a psychedelic package name and then generate malware with that name and upload it to the appropriate repository. The malicious code is then loaded by software that references the currently existing package.

knitting

robotics

• Boston Dynamics has unveiled a new humanoid robot, the successor to Atlas. Unlike the Atlas, which uses a lot of hydraulics, the new robot is all electric and has joints that can move 360 ​​degrees.
• Research robots now use AI to generate facial expressions and respond appropriately to human facial expressions. It can even predict human facial expressions and act accordingly. For example, it may smile in anticipation of a human smile.

quantum computing

• Has post-quantum encryption already been broken? We don’t know yet (nor do we have a working quantum computer). However, a recent paper proposes several possible attacks against current postquantum algorithms.
• Microsoft and Quantinuum have succeeded in building error-corrected logical qubits. The error rate of logical qubits is lower than that of unmodified qubits. Although only two logical qubits can be created, this is a significant step forward.