{"id":231,"date":"2021-03-14T01:58:14","date_gmt":"2021-03-14T01:58:14","guid":{"rendered":"https:\/\/andrejacobs.org\/?p=231"},"modified":"2022-04-11T20:24:23","modified_gmt":"2022-04-11T20:24:23","slug":"100-days-of-learning-day-5-invoking-an-openfaas-function-asynchronously","status":"publish","type":"post","link":"https:\/\/andrejacobs.org\/100-days-challenge\/100-days-of-learning-day-5-invoking-an-openfaas-function-asynchronously\/","title":{"rendered":"100 Days of Learning: Day 5 \u2013 Invoking an OpenFaaS function asynchronously"},"content":{"rendered":"\n
Photo by Lennart Schneider<\/a> on Unsplash<\/a><\/p>\n\n\n\n Here is my Log book<\/a><\/p>\n It can become an easy distant memory of how all the things work and where what lives. Especially when you end up using Note: I am mainly referencing the setup I have installed on my Mac. See my Day 0 entry<\/a> for more details.<\/p>\n How easy would it be to change the port at which we access the UI and functions from?<\/strong><\/p>\n Edit \/var\/lib\/faasd\/docker-compose.yaml\nNote: root owns all the files in my setup (so I am running these commands as root)<\/p>\n My docker knowledge is super rusty but if I recall correctly then the above ports value means, map the docker host (which would be my Ubuntu faasd instance) port 8080 to the container (gateway) port 8080.<\/p>\n I am going to see if I can map the "external" end point to be port 9999.<\/p>\n Restart faasd<\/p>\n Verify from my local machine. Note at this point just running "faas-cli list" shows it can no longer connect to port 8080. So first I need to remap OPENFAAS_URL.<\/p>\n Try and call a function.<\/p>\n Open the UI at http:\/\/192.168.64.4:9999<\/p>\n <\/p>\n It actually worked!<\/strong><\/p>\n OpenFaaS functions can be called asynchronously<\/a><\/p>\n I am going to try and write 2 functions. The first function "ping" will simulate a long running task. The second function "pong" will be called by the first function and then send out an email with the result from the first function.<\/p>\n You will notice that two URLs are available for calling the function.<\/p>\n First lets call it synchronously.<\/p>\n Ouch! I am guessing this is to do with the default OpenFaaS timeouts.<\/p>\n Checking your function’s log: Ok so it looks like there is a timeout of 5 seconds for both reading and writing.<\/p>\n A quick search later and I found this example<\/a> (and ironically it is in Python and uses sleep).<\/p>\n Edit ping.yml and set all timeouts to be 1 minute.<\/p>\n Try calling ping synchronously again<\/p>\n Ok our very slow function is working and holding up the terminal session until it completes. Take note of the Lets call the asynchronous endpoint and see what happens.<\/p>\n I read somewhere in the OpenFaaS documentation or even in the eBook<\/a> that async invocations can’t be a Ok this time by POSTing data to the function (-d "") we get a 202 response.<\/p>\n Now what? Where is the response?<\/strong><\/p>\n In Severless for Everyone Else<\/a> Alex shows us that you will need to check the logs for the queue worker on the server and use the I don’t see the returned response, but there sure is a lot of relevant info if I ever wanted to see how a function was invoked.<\/p>\n Time to move onto phase 2 and that is to have the response from ping posted to a callback endpoint using the header I am going to be installing Surprisingly that actually worked first time!<\/p>\n Until I wrote the function and then couldn’t get it to connect to the sendmail server. To fix this I needed to let the sendmail server listen on all IP addresses. I could have binded on 10.62.0.1:25 but I chose to let it listen on all interfaces.<\/p>\n At this stage the function reported the following error: … Relaying denied. IP name lookup failed [10.62.0.55].<\/p>\n Create the new pong function.<\/p>\n Edit handler.py<\/p>\n Deploy and do a test to see that email is sent.<\/p>\n Amazing! You won’t believe how much trouble I had to just try and send an email from the function. Lesson learned next time I will just deploy an out of the box email sending Docker image.<\/p>\n I should now be able to call the ping function asynchronously and tell it to callback on the pong function to email me the results.<\/p>\n First mistake was to use the 192.168.x.x address for the callback URL which the ping function don’t have access to. After changing it to the openfaas0 bridge (10.62.0.1) I could see from the logs that both functions appear to be working, but I don’t seem to receive the emails anymore … (drum roll) … they landed in my spam folder now \ud83d\ude42<\/p>\n <\/p>\n I think the hardest thing to wrap your brain around when working with VMs, containers, docker etc. is to mentally keep track of where in all this virtual space and networks the services live and what they can have access to and how data gets to them and back out to other services.<\/p>\n Initially I thought this would be a quick job of writing two functions, but instead I faced issues all over the place. At one point I even gave up on trying to send an email and instead just tried to save to \/var\/log\/something and that failed because of permissions.<\/p>\n But I persevered and learned a ton of new things along the way.<\/p>\n Something that came in very handy was being able to check the logs of the functions and the queue worker.<\/p>\n If you are going to be playing around with Photo by Lennart Schneider on Unsplash […]<\/p>\nQuick recap on some things for faasd<\/h2>\n
faas-cli<\/code> to do like 99% of the interaction while you are developing functions.<\/p>\n
\n
multipass list<\/code> to details)<\/li>\n
ubuntu<\/code><\/li>\n
Explore docker-compose.yaml<\/h2>\n
# docker-compose.yaml\n...\n gateway:\n ...\n ports:\n - "8080:8080"\n<\/code><\/pre>\n
ports:\n - "9999:8080"\n<\/code><\/pre>\n
root@faasd:~# sudo systemctl daemon-reload && sudo systemctl restart faasd\n<\/code><\/pre>\n
# For now just make changes in the current terminal session\n$ export OPENFAAS_URL=http:\/\/192.168.64.4:9999\n\n$ faas-cli list\nUnauthorized access, run "faas-cli login" to setup authentication for this server\n\n# Following my steps from Day 1 entry\n\u256d ~\/Learning\/faasd [16:40:16]\n\u2570 $ cat password.txt | faas-cli login --username admin --password-stdin\nCalling the OpenFaaS server to validate the credentials...\nWARNING! You are not using an encrypted connection to the gateway, consider using HTTPS.\ncredentials saved for admin http:\/\/192.168.64.4:9999\n\n$ faas-cli list\nFunction \tInvocations \tReplicas\niss-location \t0 \t1\nnodeinfo \t0 \t1\ncows \t0 \t1\nexpose \t0 \t1\nhelloworld \t0 \t1\n\n# Boom!\n<\/code><\/pre>\n
$ curl http:\/\/192.168.64.4:9999\/function\/expose\nHoneypot\n<\/code><\/pre>\n
Asynchronous result<\/h2>\n
Creating the ping function<\/h3>\n
$ mkdir ping\n$ cd ping\n$ faas-cli new --lang python3 ping\n$ mate ping\/handler.py\n<\/code><\/pre>\n
import random\nimport time\n\ndef handle(req):\n quotes = [\n 'Neo: We need guns. Lots of guns.',\n 'Switch: Great, the digital pimp at work.',\n 'Trinity: Dodge this.',\n "Neo: I don't like the idea that I'm not in control of my life.",\n 'Neo: There is no spoon!',\n 'Cypher: Ignorance is bliss.'\n ]\n \n time.sleep(random.randint(10,30))\n \n return random.choice(quotes)\n<\/code><\/pre>\n
$ faas-cli up -f ping.yml\n...\nDeployed. 200 OK.\n\n$ faas-cli describe ping\n...\nURL: http:\/\/192.168.64.4:9999\/function\/ping\nAsync URL: http:\/\/192.168.64.4:9999\/async-function\/ping\n<\/code><\/pre>\n
$ curl -i http:\/\/192.168.64.4:9999\/function\/ping\nHTTP\/1.1 500 Internal Server Error\nContent-Length: 31\nContent-Type: text\/plain; charset=utf-8\nDate: Sat, 13 Mar 2021 19:58:00 GMT\nX-Call-Id: eb77497d-42bd-4a2a-bfb5-90061b884dbc\nX-Content-Type-Options: nosniff\nX-Start-Time: 1615665458133139591\n\nCan't reach service for: ping.\n<\/code><\/pre>\n
faas-cli logs FUNCTION_NAME<\/code><\/strong><\/p>\n
$ faas-cli logs ping\n2021-03-13T19:57:22Z 2021\/03\/13 19:57:22 SIGTERM received.. shutting down server in 5s\n...\n2021-03-13T19:57:34Z 2021\/03\/13 19:57:34 Timeouts: read: 5s, write: 5s hard: 0s.\n2021-03-13T19:57:34Z 2021\/03\/13 19:57:34 Listening on port: 8080\n<\/code><\/pre>\n
version: 1.0\nprovider:\n name: openfaas\n gateway: http:\/\/192.168.64.4:9999\nfunctions:\n ping:\n lang: python3\n handler: .\/ping\n image: dockername\/ping:latest\n\n environment:\n read_timeout: "1m"\n write_timeout: "1m"\n exec_timeout: "1m"\n<\/code><\/pre>\n
$ faas-cli up -f ping.yml\n$ curl -i http:\/\/192.168.64.4:9999\/function\/ping\n\nHTTP\/1.1 200 OK\nContent-Length: 33\nContent-Type: text\/plain; charset=utf-8\nDate: Sat, 13 Mar 2021 20:11:01 GMT\nX-Call-Id: 83abfa46-012b-4fb3-b1c5-8399910d5669\nX-Duration-Seconds: 10.168427\nX-Start-Time: 1615666251217315311\n\nNeo: We need guns. Lots of guns.\n<\/code><\/pre>\n
X-Duration-Seconds<\/code> header. This indicates how long the function took to complete.<\/p>\n
$ curl -i http:\/\/192.168.64.4:9999\/async-function\/ping\nHTTP\/1.1 405 Method Not Allowed\nDate: Sat, 13 Mar 2021 20:13:17 GMT\nContent-Length: 0\n<\/code><\/pre>\n
GET<\/code> request and must be
POST<\/code><\/p>\n
$ curl -d "" -i http:\/\/192.168.64.4:9999\/async-function\/ping\nHTTP\/1.1 202 Accepted\nX-Call-Id: 4698d8b3-34b6-44e2-9f14-851abfeef447\nX-Start-Time: 1615666738440754292\nDate: Sat, 13 Mar 2021 20:18:58 GMT\nContent-Length: 0\n<\/code><\/pre>\n
X-Call-Id<\/code> you received when invoking the function.<\/p>\n
ubuntu@faasd:~$ sudo journalctl -t openfaas:queue-worker | grep '4698d8b3-34b6-44e2-9f14-851abfeef447'\nMar 13 20:18:58 faasd openfaas:queue-worker[14557]: [#1] Received on [faas-request]: 'sequence:1 subject:"faas-request" data:"{\\"Header\\":{\\"Accept\\":[\\"*\/*\\"],\\"Content-Length\\":[\\"0\\"],\\"Content-Type\\":[\\"application\/x-www-form-urlencoded\\"],\\"User-Agent\\":[\\"curl\/7.64.1\\"],\\"X-Call-Id\\":[\\"4698d8b3-34b6-44e2-9f14-851abfeef447\\"],\\"X-Start-Time\\":[\\"1615666738440754292\\"]},\\"Host\\":\\"192.168.64.4:9999\\",\\"Body\\":\\"\\",\\"Method\\":\\"POST\\",\\"Path\\":\\"\\",\\"QueryString\\":\\"\\",\\"Function\\":\\"ping\\",\\"QueueName\\":\\"\\",\\"CallbackUrl\\":null}" timestamp:1615666738477080588 '\n<\/code><\/pre>\n
X-Callback-Url<\/code><\/p>\n
Creating the pong function<\/h3>\n
sendmail<\/code> directly on the Ubuntu instance and use that for email delivery from python.<\/p>\n
ubuntu@faasd:~$ sudo apt install sendmail\nubuntu@faasd:~$ sudo sendmailconfig\n# Answer y to everything\n# Send a test email\nubuntu@faasd:~$ echo "Hello world!" | sendmail -v email@address.com\n<\/code><\/pre>\n
ubuntu@faasd:~$ sudo nano \/etc\/mail\/sendmail.mc\n# Comment out the 2 lines where 127.0.0.1 is being used (by adding dnl)\n\ndnl DAEMON_OPTIONS(`Family=inet, Name=MTA-v4, Port=smtp, Addr=127.0.0.1')dnl\ndnl DAEMON_OPTIONS(`Family=inet, Name=MSP-v4, Port=submission, M=Ea, Addr=127.0.0.1')dnl\n\n# Save and exit\n# Generate config file\nubuntu@faasd:~$ sudo m4 \/etc\/mail\/sendmail.mc > \/etc\/sendmail.cf\n# Restart sendmail\nubuntu@faasd:~$ sudo systemctl restart sendmail\n\nubuntu@faasd:~$ netstat -tuna # <-- I love this, easy to remember tuna\n# Install netstat using: sudo apt install net-tools\ntcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN\n<\/code><\/pre>\n
ubuntu@faasd:~$ sudo nano \/etc\/mail\/access\n\n# Uncomment the line (this will relay 10.x.x.x)\nConnect:10 RELAY\n\nubuntu@faasd:~$ sudo -i\nroot@faasd:~# sudo makemap hash \/etc\/mail\/access.db < \/etc\/mail\/access\nroot@faasd:~# systemctl restart sendmail\n<\/code><\/pre>\n
$ mkdir pong\n$ cd pong\n$ faas-cli new --lang python3 pong\n$ mate pong\/handler.py\n<\/code><\/pre>\n
import smtplib\nfrom email.message import EmailMessage\n\ndef handle(req):\n msg = EmailMessage()\n msg.set_content(req)\n msg['Subject'] = 'ping result'\n msg['From'] = 'ubuntu@faasd'\n msg['To'] = 'you@youremail.com' # This should be a secret\n\n s = smtplib.SMTP('10.62.0.1') # This should be an environment variable\n s.send_message(msg)\n s.quit()\n\n return "done"\n<\/code><\/pre>\n
$ faas-cli up -f pong.yml\n$ curl -d "Hello world" http:\/\/192.168.64.4:9999\/function\/pong\ndone\n<\/code><\/pre>\n
Playing ping pong<\/h3>\n
$ curl -d "" -i http:\/\/192.168.64.4:9999\/async-function\/ping \\\n--header "X-Callback-Url: http:\/\/10.62.0.1:9999\/function\/pong"\n\nHTTP\/1.1 202 Accepted\nX-Call-Id: 6286dcb7-46f7-4dfd-a8e7-d6c495d99db6\nX-Start-Time: 1615684430227839637\nDate: Sun, 14 Mar 2021 01:13:50 GMT\nContent-Length: 0\n\n# Repeated this a few times\n<\/code><\/pre>\n
Conclusion<\/h2>\n
ubuntu@faasd:~$ sudo journalctl -t openfaas-fn:FUNCTION_NAME\nubuntu@faasd:~$ sudo journalctl -t openfaas:queue-worker\n<\/code><\/pre>\n
faasd<\/code> then I highly recommend you buy a copy of Severless for Everyone Else<\/a>. I find myself coming back to the book time and time again.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"