AI in my pocket

By    John Garner on  Saturday, May 13, 2023
Summary: A novel AI topic that is trending, is around the porting of foundation models like Llama on to Google Pixel phones. This also maps to the leaked Google Memo about the threat of open source to their general 'moat model'.

As discussed in the previous post, AI is a nascent and revolutionary technology. Why, well, it looks set to dramatically change and impact society, the economy and our jobs as we know them.

A leaked memo from Google gives us a glimpse of how even the major companies fighting for the top position may not be in such an enviable position as we thought. It also explains how many major IT and consultancy companies seem a bit lost, or at least on the back heel with their offerings.

One thing I take away from the leaked Google 'Jerry Maguire 2023 memo' is that as consumers we should, in the short to medium term, start seeing options on our own personal equipment. The resources needed for such "assistant and chat" AI systems may mean it first starts with powerful laptops before our smartphones have the appropriate hardware. Or it may be through hybrid versions that can manage certain things locally and offload more power hungry tasks to a server. Smartphones have several blockers including having the equivalent of several physical GPUs, as well as the memory wall for all the params that need to be readily accessible, an issue that servers don't have but all current smartphones would come up against.

As I manage servers myself, I could easily see myself creating a server part of a phone assistant, just for my family and I, for example. I'm sure some brilliant developers will think of ways to do this, whether it is hybrid or some other way. It's why the memo makes a lot of sense. While Google and OpenAI are in a battle for dominance. Wouldn't it be great if there was a 3rd option that was great for society as well rather than the just the Tech giants?

It also means that choosing the hardware on your phone may be more and more important. For example, few phones other than the Google Pixel have anything like the necessary hardware for some of these AI / Chatbot / Assistant etc functions. Even then, it is mainly for well-known functions like "voice to text". It is a sign Google has been working on these types of evolutions for quite some time, anticipating the required hardware.

After the timid market response with Google Glass, it would be understandable to imagine a lot more caution in unwrapping their AI tools to the public before they are ready. OpenAI launched theirs with far less regard for risks, 'à la Facebook', launch it as soon as possible and fix bugs asap.

I used Gmail shortly after it launched and I remember the beta label that lasted for years. You would have thought Bard could also have used the same beta label, you know, like OpenAI does!

Time will tell whether the leak of Llama developed by Meta and currently being used by several open source AI projects will end up giving Meta a considerable advantage compared to the closed door policy of Google and OpenAI. Google's research work over the past years that was shared through official publications was one of the key drivers of where we stand today with AI, notably this famous study from Google (pdf).

But in recent months, Google has overhauled its AI operations with the goal of launching products quickly, according to interviews with 11 current and former Google employees.

Google, like OpenAI, has changed gears in recent months, taking cues from one another in terms of major strategic decisions and roadmaps for AI related topics.

But as per the famous leaked memo, so too, have we seen passionate open source developers and other entrepreneurs diving into the opportunities AI promises.

There are already developers who have got Llama / foundation models to work on a Google Pixel phone, be it at 5 tokens per second.

The ability of a phone to provide hardware support for AI related functions and activities is going to become an interesting selling point. As per above it is unlikely that phones will be able to cover GPU needs and high memory capacity will also be an interesting plus.

Article written by  John Garner

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts

Check out the most recent posts from the blog: 
Sunday, September 24, 2023
The reliability & accuracy of GenAI

I question the reliability and accuracy of Generative AI (GenAI) in enterprise scenarios, particularly when faced with adversarial questions, highlighting that current Large Language Models (LLMs) may be data-rich but lack in reasoning and causality. I would call for a more balanced approach to AI adoption in cases of assisting users, requiring supervision, and the need for better LLM models that can be trusted, learn, and reason.

Read More
Saturday, September 23, 2023
From Chatbots to Reducing Society's Technical Debt

I discuss my experience with chatbots, contrasting older rules-based systems with newer GenAI (General Artificial Intelligence) chatbots. We cannot dismiss the creative capabilities of GenAI-based chatbots, but these systems lack reliability, especially in customer-facing applications, and improvements in the way AI is structured could lead to a "software renaissance," potentially reducing society's technical debt.

Read More
Friday, June 16, 2023
The imbalance of power in the AI game: in search of the common good

The article discusses the contrasting debate on how AI safety is and should be managed, its impact on technical debt, and its societal implications.
It notes the Center for AI Safety's call for a worldwide focus on the risks of AI, and Meredith Whittaker's criticism that such warnings preserve the status quo, strengthening tech giants' dominance. The piece also highlights AI's potential to decrease societal and technical debt by making software production cheaper, simpler, and resulting in far more innovation. It provides examples of cost-effective open-source models that perform well and emphasizes the rapid pace of AI innovation. Last, the article emphasises the need for adaptive legislation to match the pace of AI innovation, empowering suitable government entities for oversight, defining appropriate scopes for legislation and regulation, addressing ethical issues and biases in AI, and promoting public engagement in AI regulatory decisions.

Read More
Thursday, June 1, 2023
Japan revises copyright laws for AI

Japan has made its ruling on the situation between Content creators and Businesses. Japanese companies that use AI have the freedom to use content for training purposes without the burden of copyright laws. This news about the copyright laws in Japan reported over at Technomancers is seen as Businesses: 1 / Content Creators: 0 The […]

Read More