Would be nice if we could sign in and use our own custom GPTs as the default assistant, especially since they have memory now.
So much yes to this. I’d extend this by making the LLM and speech-to-text algorithms configurable.
The biggest problem I’m having with my R1 is the long prompts, which don’t usually prompt a response.
I’d be extremely happy if I could configure my r1 to use GPT-4o and Whisper for these purposes. I’m willing to enter my API key and pay for my usage - which wouldn’t be a problem at all. It’d come with multiple benefits IMO:
- I pay for my prompts, so rabbit can subsidize that.
- The quality would have a significant jump.
- My rabbit would work in many languages, which would help me A LOT with language training and using different languages.
GPTs are on the roadmap so should be able to in the future.
That would be cool. But why wouldn’t you just use your phone then? Isn’t the point of this for it to be a LAM and not an LLM? If you could use your own LLM, how would that be able to translate into the contextual awareness of how to use the LLM query going into 4o to call LAM tasks and integrations?
I built a rudimentary smart home GPT - something that can take normal human language like “damn it’s hot in my room” and recognize that I want to turn down the thermostat. That’s what LAM is supposed to do. Since the systems are disparate and I’m not going to spend a year writing Alexa Tasks - the actual doing is limited. I have fed it all of my smart devices, so it knows how to take my random musings and convert those into commands to various devices in my house - but that integration just isn’t doable for me. I have however hacked around it - I have my GPT convert my LLM responses that translate into smart home commands and then i have it speak to Alexa audibly. Works about half the time as Alexa is finicky with computer generated voices trying to activate it. It is kind of fun though to have GPT talk to Alexa and make stuff happen.
This would be cool, but I think getting an answer on R1 might involve more than one LLM. It might be hard to control the quality of responses or improve them if the LLMs being used are variable in the backend and always subject to change.
Rabbit R1 can do both LLM and LAM. If it’s possible to use a more custom tailored GPT as the LLM assistant, without hurting the LAM side, then I think it should.
My take on this - but I’m just thinking about how this could work, because it would be awesome, and it is possible I’m way off base with my thinking.
I guess if you wanted to use an LLM for conversational AI against 4o (like your phone) that could work - but it would never be able to then work the LAM tasks (not that any exist rn). Without the ability to utilize that LLM to understand what you want to do and translate it into “commands” against a LAM model, they would live in separate boxes that wouldn’t talk to each other.
When I went through the process of creating my own TAM (tiny action model lol), that was always the trick. While it was mind numbing to set up all the smart devices in my house and what they can do, I was able to teach a GPT to analyze my prompts and return a list of actions it would take in response. Something like “it’s movie time” - it would create a list of actions that it figured out it should do - turn on the tv, set the receiver and volume, close the curtains, set the lights, turn on the tv backlighting, etc, etc. So I trained it to break down my crazy into actionable things to make me happy. The gap then is how, in a disparate world of IoT devices, do those commands go from just being a list to actually being processed.
In my mind that’s always what this theory should be doing - but it’s that bridge from understanding intents and appropriate actions, to making those actions “go” that’s the fundamental problem R1, I thought, was going to solve.
Now I just have my GPT barking commands to an Alexa. It’s kind of fun when they don’t get along. But with a solid 20% success rate I’m not holding out much hope for my own TAM :).
Let me know if you were thinking about this a different way - would love to learn more about what you are thinking.
@jwc What a creative solution; voice to bridge AIs vs. an API.
Desperate times and determination result in horrifically awful but elegantly hacked solutions lol. All will probably be solved with our friends in Cupertino - now I just have to unfortunately switch over to HomeKit for all my stuff - which is annoying.
The biggest issue is just how fragmented my stuff is - and i suspect everyone has the same issue who wants to do what i want to do. Smart Life, smart things, zigbee this, alexa that, HomeKit this, yale that, blah blah. Hopefully Amazon gets their act together - but the beta i was supposed to see 6 months ago still hasn’t seen the light of day, so I’m not holding out much hope.
Along with that is the frustration of actually teaching something about all your devices and where they are and what they can do - that was the most time consuming part of things. The matrix GPT built though was pretty cool.
And thanks! First time I tried it and GPT started barking at Alexa and it worked, I was even impressed with myself! For about a minute, and then it stopped working lol.
You must have the patience of Job to try this. Kudos!
I’d like to think that’s true - but it is more about the determination to bending this AI crazy into my life that drives it. Anyone who has used GPT extensively, which I’m guessing most people actually reading this crazy diatribe have, knows how horribly frustrating it can be. The science and art of talking AI into doing what you actually want it to do is like coding back in the day - before everything was a configuration based, IDE cut and paste auto-fill, or google search away. You got to love working on fun problems that you can’t just google to move forward on, because nobody has done it. Also, it’s even more fun because the AI itself doesn’t know how to do it lolz.
Just a usability question for you - as I love your thinking here. Why would you, in the case you are discussing, want to use the Rabbit for this? I’m totally missing your context of what problem you are trying to solve here. Would love to learn what you are thinking!
I support the request; it would be great to be able to choose the default assistant and even add our own API.
Frankly speaking, if r1 would use other AI it would become redundant to smartphones. Instead of that it would be suggested that Rabbit could implement their AI in order to allow further interactions with more third part apps.
id very much like the ability to install ChatGPT so i can maximise the benefit of my Plus sybscription, and it would be awesome to have it on the Rabbit - a PPT ChatGPT is what is missing in the world!
Here are some use cases for using Rabbit R1 to converse with OpenAI’s custom GPTs.
Use Case 1: Smart AI-Assisted Tutor in your pocket.
Description: On a specific topic, such as learning about the geography of a country. The smart AI-Assisted Tudor assesses your current knowledge and suggests activities to increase it. Activities include Q&A, solving a puzzle, watching a video, taking a trip, accessing Google Maps, etc.
Use Case 2: Business Consultant in your pocket.
Description: For a specific business, help identify a business problem and make suggestions to resolve the problem.
Implementation: Rabbit R1 converses with custom GPTs and recognizes, based on the conversation, what custom GPT to call next and what file to input into the custom GPT for a simple form of Retrieval Augmented Generation (RAG).
Here is an example of a simple loop:
Start ==> GPT1 ==> File 1 saved in Rabbithole, auto call GPT2
File 1 ==> GPT2 ==> File 2 saved in Rabbithole, auto call GPT3
File 2 ==> GPT3 ==> File 3 saved in Rabbithole, auto call GPT4
…
File n ==> GPTn ==> Filen saved in Rabbithole and auto call GPT1
Hopefully, the LAM can be trained to listen in on the conversation and understand what custom GPT to call next. This framework could be used for almost every use case I can think of.
Thoughts?