ChatBot 4 - Sioni on the Web

Leat House

After a long break from ChatScript I now have enough to make Sioni vaguely interesting - to me at least.

Here he is: sioni.online

Status Update

I started out trying to find a way to build a conversational agent that would capture the life and times of someone like my grandmother. That way people could sit down and have a chat with a distant relative and find out about their lives in a much more engaging way than reading some facts about where they were born and seeing some photographs about where they lived.

How are things going so far? Here's a sanity check:

• Is it engaging enough?
  Yes, it's not bad. It could be much more engaging if I put more time into the script. However, ChatScript is a great tool for bots, there's no doubt.
• Is it possible to simply feed a bunch of stories and facts into a repository and have the bot use that for its memory of events for conversions?
  No, that's not what ChatScript does in it's current form. Although the recent extension of ChatScript to call out to other sources of truth using REST APIs is worth pursuing. Commercial bots obviously do this to some extent based on knowledge bases.
  Next blog post maybe. 
• Is the experience rich enough?
  It's ok but it would be useful if the bot was able to use URL's in the script to display additional content or play audio files.  Another blog post.

What did I learn?

Events not Facts

I started with a mind-map of Sioni's world. Being as ChatScript is driven by topics, structuring the mind-map by topics is a good idea. I started with topics for friends, family and places etc. My initial approach was to have a whole bunch of facts as rules. This probably reflects my Prolog programming thought process. However, this approach doesn't result in particularly entertaining conversations! It is much better (as the ChatScript docs suggest) to have your bot drive the conversation. So I ended up with topics for specific life events driven by the bot. If you're building a bot based on your life events (as Sioni is) this is a much more natural way to remember your past and structure your bot. When we catch up with family and friends and talk about the past, we almost always talk about particular events - the memorable ones are often the most emotional too: funny or sad or significant in other ways. Once I figured out that was my best approach, the flood gates opened and the memories of particular events came in thick and fast: The day the onion seller came to the village, that time when I rode my bike into Annie May, the times we went to the big swing past the sand-banks and threw coins into the wishing pond, my first day at school, picking pees in the garden for dinner etc. etc.

If you have a chat to Sioni, you'll see these events come through along with some of my fact-based style weaved in. I haven't gone back and cleaned up the scripts - this is an experiment in bot creation and not a fully complete and consistent bot.

AWS Server

sioni.online is hosted in Amazon (AWS). If you're a bit technical, setting up the Linux server in AWS is straight forward. In fact, it's a lot easier than setting it up on the Mac. Look at "ChatScript Amazon Server.pdf" in the documentation directory.
I used the BETTER PHP files and modified them to call Sioni.

Topics and Behaviour

Because ChatScript remembers where you left off, its behaviour can be confusing when you're trying to develop more script. You need to rebuild and reset to get a clear idea of what it's doing. Initially, I was confused about why a certain topic was being executed and why. In simple terms, ChatScript stays in its current topic as long as there are things left to say and what you've typed matches a rule. If you type something that can't be matched, it will find a typic which will match and then find a rule within that topic. It's pretty obvious really but the behaviour when you're developing can be confusing.

Sublime Text Editor

There are now themes for ChatScript. Syntax highlighting makes writing ChatScript much easier!

ChatScript files

While ChatScript ships with example bots, creating a new bot by copying an existing one can lead to some confusion. I ended up copying quibble.top, introductions.top, childhood.top, keywordless.top, simplecontrol.top and the entire quibble Eliza file set. Update introductions for your bot, add a bunch of things to keywordless.top that makes sense for your bot, and update the simplecontrol.top to make sure it has the right topics are called in the control loop. Most of the text for Sioni is obviously in childhood.top.

ChatBot 3 - Getting started with ChatScript

A random picture of Alan Turing
for inspiration

Having lost faith in AIML as a platform to capture life history, I've turned to ChatScript by Bruce Wilcox. Amazing.

Bruce explains why his approach (ChatScript) is an improvement on AIML based ChatBots in the Speaker for the Dead paper:

"AIML is based purely on pattern matching literal words. It compares scripted patterns against the input until it finds a match and then presents the prerecorded response. This is incredibly limiting, and AIML has excruciatingly over-precise pattern match capabilities. One can not easily author a lot of content, nor can it get anywhere close to finding the meaning of an input.

I've played with it for a while now and am very impressed!
There are a couple of major functional differences between ChatScript and AIML to note:

• ChatScript is written in plain text and not XML. It's a computer scripting language and to my mind, much easier to write, test and debug than AIML.
• It supports the notion of concepts. That is, you can test to see if a word fits the concept rather than simply a text match. It comes with built in set of concepts arranged in a taxonomy and you can simply add your own as well. 
• The pattern matching is MUCH more flexible that AIML.
• Groups of rules are arranged in topics. Topics can be selected through a list of keywords or concepts. ChatScript will then select rules in that context. For example, a concept such as my my_dog, may list keywords such as ( dog Sarah puppy ) as triggers to enter this topic.

But first, I'm going to list the basics of how to get started. This section is all covered in Bruce's documentation but is spread across a few documents. What follows is my "cheat sheet" for getting started.

How to get the source:

Go to the sourceforge project here:

http://sourceforge.net/projects/chatscript/

Download and unzip the latest zip file, for example: ChatScript-4.5.zip

Running on a Mac?

If you're running on a Windows PC or Linux, then skip this section.

Sadly, if you're running on a Mac then you'll need to compile the code. It's not difficult - follow thee steps:

Download Xcode for Mac from here:

https://developer.apple.com/technologies/tools/

If you're not a Mac developer you'll have to register first.

Run Xcode and download the command line tools. You'll find this option in

Preferences>Downloads.

Browse to the base directory of your ChatScript files and in a command line window and run:

g++ -lpthread -funsigned-char src/*.cpp -O2 -oMacChatScript 2>err.txt

Then try and run the server as follows:

./MacChatScript local

You should see the server start and have a promp on the command line. Press enter and see if it’s working (talking). If it is, you'll be talking to Harry - the default bot.

Note: If you get stuck and just want a working Mac binary let me know and I'll send you mine. 

In the long run it's worthwhile being able to build your own as Bruce releases new versions quite frequently.

Running on a PC?

There is a EXE file provided with the download. Simply run that as follows:

ChatScript.exe local

Running on Linux?

32 bit and 64 bit binaries shipped with the zip file. You'll need to make them executable:

chmod +x ./LinuxChatScript32

./LinuxChatScript32 local

or 

chmod +x ./LinuxChatScript64

./LinuxChatScript64 local

Documents

Take a look under the DOCUMENTATION folder and you'll see a comprehensive collection. It's not easy to navigate at first, but you'll quickly learn where the relevant information is.

Create a new bot

The first think you'll want to do is create a new bot - your bot.

The document "Bot Harry - basic bot.pdf" covers this in some detail.

Essentially you have to make a copy of HARRY and change all the Harry text references to your own. Mine's Sioni so I'll stick to that below. 

So...

In the folder RAWDATA copy the HARRY folder and rename it. Use this as the base for your new bot.

Change the name at the bottom of introductions.top

Change Harry in simplecontrol.top to Sioni

In main folder, copy filesHarry.txt and rename filesSion.txt

Edit this file and change the path for Harry's files to Sioni's files.

I've reused childhood.top by removing the content and stating again. This is fine for Sioni as it is all about his childhood. If you wanted a file with a different name simply rename it.

Common Rules

The complete ChatScript language is quite extensive and complicated. However you can build a complete chatbot with just a subset of simple commands.

# This is a comment. We're going to have a topic about my dog Sarah.

# If the user is asking about dog or Sarah this topic will 

# be active, or if there's no input anything with a t: may be 

# spoken by Sioni.

topic: ~about_dog ( dog Sarah )

# Sion will speak this once as a conversation starter.

t: We have a dog called Sarah.

# if the answer includes 'do you' Sioni will say the following text.

    a: ( do you ) Yes. What do you want to know about her?

# Again, random question from Sioni

t: Do you know what Sarah likes to eat?

# If you type anything with 'what' or anything that's the concept 'no' then Sioni will tell you she likes rabbits.

    a: ( [ what ~no ] ) She likes to catch and eat rabbits the most but also quite likes squirrel.

# If you guessed rabbits.

    a: ( rabbit ) Yes! She likes rabbit!

# If no match, capture input and write it back in another sentence.

    a: ( _* ) Oh I don't think she likes that... '_0 really?

Simple Build and Test cycle

You'll be editing code and wanting to test it quite frequently. It's important to remember that when you exit ChatScript and login again, ChatScript will continue the session where you left off. Just to be clear, you don't really login to ChatScript, rather you tell it who you are and it uses that to maintain a session state. But when you're writing and testing code, starting where you left off can be very confusing. It's better to have a clean slate. Occasionally your code will cause a catastrophic error which messes up the ChatScript state so badly that you have to rebuild the knowledge base. This is addressed below.

Note: I'm using local mode, hence the 'local' for the startup argument. This simply allows you to interact with ChatScript at the command line. You can run it as a server and use a web front end but for the purposes of developing the scripts themselves, the command line interaction is much more effective. I'll talk about other front-ends in some later post - honest.

Typical Steps are:

Edit some code the run the server, provide a user name (anything) and then type:

:build sioni

Test your new rules, debug and fix bugs and try again... etc.

to reset the rules and start the conversation afresh do this when you're logged on:

:reset user

I do this pretty much every time I want to retest by code.

Complete Clean Build

In the event of a bad crash, rebuild the whole system as follows:

Delete the contents of the following two directories: USERS and TOPIC

Then start the ChatScript again and run:

:build: 0

:build sioni

This builds the ChatScript base again and adds your bot. You have a fresh new bot with all the old history cleaned out!

Unit Tests

If you're a developer you'll be familiar with the notion of unit test. ChatScript supports this through a special comment type before a pattern. The comment is used to test the pattern that follows.  For example:

#! Does Sarah like rabbits?

?: ( Sarah like rabbits ) Yes she does.

It's worth adding these unit tests as you go.

Next time: I'll show you a much more functional Sioni and let you know what I've learnt along the way about constructing a chatbot using ChatScript.

ChatBot 2 - Sioni at Pandorabots

Llanllwch - Sion's home town

AIML is a common approach for ChatBots. I take a look at Pandorabots and and see if it meets my needs.

Sioni the Person

I set out to build a ChatBot that would capture facts about a person such as place of birth, parents and stories from their life etc. A ChatBot that could provide much more than a simple family tree and some photos.
I need subject to test with so I've decided to pick myself.


To include all my personal history and stories is impractical, so I'll restrict the test to a short period of my life in a small village called Llanllwch in Carmarthenshire, Wales. I was about 3 years old when we moved there and left when I was about 7 years. That is, a short, contained period of my life in a single location, with a very limited number of friends, at an age where I only remember key events and short stories - perfect.

When we moved to Llanllwch, I was a Welsh language speaker. My mother used to call me Sioni (Johnny) or Sion (John). So I've picked this name for my ChatBot. Sioni Llanllwch. However, I'll assume for the time being that Sioni speaks English. Actually he does, so that's ok. However, there will (necessarily) be a smattering of Welsh terms, names etc that featured in Sioni's life. This will make things more interesting.

Sioni at Pandorabots

There is a Standard created to capture a ChatBot's knowledge base, it's AIML. The Artificial Intelligence Markup Language. A term (I now realise) that implies a broader range of capabilities than it actually covers.

There are a number of websites where you can create your own ChatBots based on AIML.

I've picked Pandorabots - it's free and easy to use.

Note: If you want to replicate my activities, then sign up and create a ChatBot.

The first thing I did was to remove all the default AIML files. I'm not sure if this is wise, but there are a lot of default files that contain information I simply would not have known as a kid!

I started using the "Train" option but this is rather laborious. Not only that but it's not a great way to capture a lot of information in a way that can be edited later.

You could edit the AIML files directly, but this requires a very good understanding of the XML syntax and the result is sure to be error prone. Also, the XML structure will simply get in the way of capturing a simple dialog.

To being with, I've opted for uploading my own text file using the Dialog to AIML Parser. This simply takes a relatively freeform question and answer structure and converts it into AIML which you can then load into your ChatBot.

The free form text I've used is here.
When converted to AIML, it looks like this.

Upload to your Pandorabots by going to the AIML Tab for the ChatBot properties.
Remember to publish your ChatBot after uploading new files.

You can talk to Sioni here.

Here's an example chat I had with Sioni when I knew exactly what to ask:

Me: Did you ride a bike?
Sioni: Yes, after a while we had bikes and we used to ride up and down the road to the railway bridge and back. Sometimes we went up Allt y Crap hill and rode down at high speed.

Me: Where did you live in Llanllwch?
Sioni: We lived in Leat House. It was a modern house designed by my father.

Me: Do you have a brother?
Sioni: I have a brother called Simon.

Me: What's your mother's name?
Sioni: Ileana Jones or Ileana Maesydd to some who knew here when she was younger. 

So far so good! However, if you don't match the questions exactly, the answers are repetitive and less than engaging:

Me: Did you have a cat?
Sioni: I have no answer for that.

Me: Did you have a dog?
Sioni: I have no answer for that. 

You can include wild cards which carry the matched part of the sentence through to the ChatBot answer but this isn't that useful when representing a knowledge base.

For example,
Me: "Do you like *"
Sioni: "Yes I like *"

More advanced AIML

To take advantage of AIML's power, you need to leave the simple text format and jump in to direct XML editing of the AIML.

In simple terms it has patterns to match on and templates for text output.  It supports recursive text matching through a key-word <srai>. This one construct is used to create numerous synonyms for words or phrases and provide a seemingly more intelligent agent.

A more elaborate AIML file can be found here.

Dialog example:

Me: did you have a bike?
Sioni2: After a while we had bikes and we used to ride up and down the road to the railway bridge and back. Sometimes we went up Allt y Crap hill and rode down at high speed. 

Me: Tell me about Allt y cnap hill
Sioni2: We used to watch the hunt from the top of the hill. I remember watch the fox get away. That was good to see. 

Me: and where else did you ride your bike?
Sioni2: I had a bike and when I learnt to ride I rode straight in to Any May! 

Me: what was your father's name?
Sioni2: Donald Jones, or Jones the Architect. 

Me: did you have a brother?
Sioni2: 1 I have a brother called Simon. He's a year younger than me. 

Me: did you know a tramp?
Sioni2: There was a tramp who lived in Allt y cnap woods sometimes. 

Me: tell me more about the tramp
Sioni2: When I was a kid there were lots of them arround. They used to go from one parish to another and get food from the vicar. 

Conclusion

This is not what I'm after...

To be fair, AIML does allow a whole host of additional functionality which amounts to more flexibility in pattern matching for pseudonyms etc. I could have made Sioni a lot more convincing with time and effort. Maybe that's exactly what's required.

There's no doubt that Pandorabots have also done exceedingly well at the Loebner prize over the years  - see their blog for an overview. AIML based ChatBots can be good - very good!

So what's my problem?

To begin with, AIML based systems follow a pattern matching approach and do not track context or represent knowledge in a way that can be queried in an ad-hoc manner. It's a bit like teaching a horse to count - see clever Hans.

The other problem with this approach is the input data needs to be highly structured. Ideally some means of inputing free form text would be good!

There's plenty of information on the web on the pros and cons of AIML. Take a look here.

So for now, I'll leave AIML based ChatBots and take a look at what other approaches have been tried and how they differ from (completely) AIML based systems.

ChatBot 1 - My Grandmother's a ChatBot

Mechanical Turk

The Idea

It stuck me some time ago that the amount of knowledge we have of our grandparents and great grandparents is very limited. If you're lucky, you might know where they were born, where they lived and went to school, what job they had, who they married and who their children were. That's not a lot of information! You might get the odd story or two, anecdotes about their childhood etc.

You could record this information in a document, or... you could bring it to life through a conversational agent of some sort!

So... rather than aim for a complete biography of someone's life, would it be possible to capture a reasonable set of facts, stories and events that could be used to host a conversation with that "person"? A conversation that would not be too much different from one you might have had with your grandmother anyway. Light on detail, possibly contradictory, and often confusing - but informative, possibly funny and hopefully interesting.

Would this be an useful way to capture someone's life history? Would it help future generations understand a bit about their ancestors in a more engaging manner than reading a document? Is it an useful way to teach history? Is the current technology up to the task and freely available for the masses? Is it also possible to create a fictitious interviewer to gather this information in the first place? Does the information need to be codified by experts before making it available to a computer program? Is it possible to automate this activity and provide a system that was fit for commercial use? etc. etc.

This is not a new topic and much has been done to address these questions over the last few decades. I'm aware of some of this work but would like to find out more. I'm also after feedback and guidance from others who have explored this trail before.

I'l set off to explore these ideas with the aim of constructing various implementations from the freely available software and see if I can answer some of the questions I've raised above. This is meant to be a fun project and not an authoritative treatise on artificial intelligence!

Background

The idea of a computer or machine that embodies a person's knowledge and abilities has been around for quite a while. The term ChatBot is commonly used today.

The Turk (or Mechanical Turk) was an eighteenth century creation that used a real person inside a cabinet to provide the allusion of an intelligent machine that played chess. The appeal of such a contraption has remained with us ever since.

In the 1950s Alan Turing devised the Turing Test to determine if a machine could be regarded as intelligent. Actually, what he proposed was something slightly more subtle, a test to tell if a human could tell the difference between a human and a computer through a dialog with both via a screen and keyboard. This gets around the problem of defining "intelligence" but is rather subjective.

A competition is held each year along the same lines: it's the Loebner Prize.

There are other examples of computers beating humans at their own game. Watson, the IBM computer program that won the quiz show Jeopardy is one such impressive machine along with Deep Blue, the chess computer - a formidable challenge to the most accomplished chess masters.

Interestingly, Watson's creators were concerned that the Jeopardy staff would turn the game in to a Turing Test by exploring the  deficiencies of Watson. More on this problem later, but a clear indication that their aim was not to mimic a human Jeopardy player, but simply to create a machine to win Jeopardy.

But now we're digressing - while Watson and Deep Blue are examples of very smart computers, the capability we're after relates specifically to programs that can act as a convincing proxy for a real person. Unless all your grandparents did was quiz-shows and play chess, we'll need to look elsewhere.

So... have people thought about capturing an individual's knowledge before? Yes, many as it turns out...

Bruce Wilcox, a winner of the Loebner Prize, wrote a short paper titled: Speaker for the Dead. While a slightly macabre twist on the topic at hand, the paper explores the issues directly and provides insights into the problems and limitations of commonly used ChatBot technologies.

There are also many examples of specific individual ChatBots on the web:

John Lennon is a good example to start with and a whole page of others here.

There's lots more that can be said about ChatBots on the web, but I'll move on to looking at the most popular ChatBot technology next and see how they stack up.