Wednesday, July 23, 2014

Embedding python in bash scripts

As a software development consultant, I do a lot of bash scripting.  I can do a lot of really creative things using nothing but bash and the binutils at my disposal, but sometimes I'll come across something that's just easier to do in a higher level scripting language.  Enter python.

Conversely, there are a lot of things that are just easier and more straight forward to do in bash, so writing everything in straight python may be more work than it's worth.

Here's a quick posting to describe how you can embed some python code into your bash scripts and get the best of both worlds.  Note: just as a heads, up, the examples in this posting are quite contrived.

Calling python from bash is easy.  You simply use python's '-' argument and pipe in your python code.  I typically wrap my python code in a bash function.

#!/bin/bash

function current_datetime {
python - <<END
import datetime
print datetime.datetime.now()
END
}

# Call it
current_datetime

# Call it and capture the output
DT=$(current_datetime)
echo Current date and time: $DT

You can also pass data into your embedded python script.  I do that using environment variables:

#!/bin/bash

function line {
PYTHON_ARG="$1" python - <<END
import os
line_len = int(os.environ['PYTHON_ARG'])
print '-' * line_len
END
}

# Do it one way
line 80

echo 'Handy'

# Do it another way
echo $(line 80)

My usual use-case for doing this is if I'm extending someone else's bash scripts and have to 'go off the reservation' a bit.  Sometimes I'm updating an existing 'legacy' script and need to look up some data... maybe do a REST call or something.  Here's an example bash script that uses curl to call a REST service to get some weather data.  Then is passes the raw JSON response to an embedded python script to interpret and format the results:

#!/bin/bash

function format_weather_data() {
PYTHON_ARG="$1" python - <<END
import os
import json

json_data = os.environ['PYTHON_ARG']
data =json.loads(json_data)
lookup = {
    '200': 'thunderstorm with light rain',
    '201': 'thunderstorm with rain',
    '202': 'thunderstorm with heavy rain',
    '210': 'light thunderstorm',
    '211': 'thunderstorm',
    '212': 'heavy thunderstorm',
    '221': 'ragged thunderstorm',
    '230': 'thunderstorm with light drizzle',
    '231': 'thunderstorm with drizzle',
    '232': 'thunderstorm with heavy drizzle',
    '300': 'light intensity drizzle',
    '301': 'drizzle',
    '302': 'heavy intensity drizzle',
    '310': 'light intensity drizzle rain',
    '311': 'drizzle rain',
    '312': 'heavy intensity drizzle rain',
    '313': 'shower rain and drizzle',
    '314': 'heavy shower rain and drizzle',
    '321': 'shower drizzle',
    '500': 'light rain',
    '501': 'moderate rain',
    '502': 'heavy intensity rain',
    '503': 'very heavy rain',
    '504': 'extreme rain',
    '511': 'freezing rain',
    '520': 'light intensity shower rain',
    '521': 'shower rain',
    '522': 'heavy intensity shower rain',
    '531': 'ragged shower rain',
    '600': 'light snow',
    '601': 'snow',
    '602': 'heavy snow',
    '611': 'sleet',
    '612': 'shower sleet',
    '615': 'light rain and snow',
    '616': 'rain and snow',
    '620': 'light shower snow',
    '621': 'shower snow',
    '622': 'heavy shower snow',
    '701': 'mist',
    '711': 'smoke',
    '721': 'haze',
    '731': 'sand, dust whirls',
    '741': 'fog',
    '751': 'sand',
    '761': 'dust',
    '762': 'volcanic ash',
    '771': 'squalls',
    '781': 'tornado',
    '800': 'clear sky',
    '801': 'few clouds',
    '802': 'scattered clouds',
    '803': 'broken clouds',
    '804': 'overcast clouds',
    '900': 'tornado',
    '901': 'tropical storm',
    '902': 'hurricane', 
    '903': 'cold',
    '904': 'hot',
    '905': 'windy',
    '906': 'hail',
    '950': 'setting',
    '951': 'calm',
    '952': 'light breeze',
    '953': 'gentle breeze',
    '954': 'moderate breeze',
    '955': 'fresh breeze',
    '956': 'strong breeze',
    '957': 'high wind, near gale',
    '958': 'gale',
    '959': 'severe gale',
    '960': 'storm',
    '961': 'violent storm',
    '962': 'hurricane',
}

print "Current temperature: %g F" % data['main']['temp']
print "Today's high: %g F" % data['main']['temp_max']
print "Today's low: %g F" % data['main']['temp_min']
print "Wind speed: %g mi/hr" % data['wind']['speed']
weather_descs = [lookup.get(str(i['id']), '*error*') for i in data['weather']]
print "Weather: %s" % ', '.join(weather_descs)

END
}

WEATHER_URL="http://api.openweathermap.org/data/2.5/weather?q=Cincinnati,OH&units=imperial"

format_weather_data "$(curl -s $WEATHER_URL)"

Hope you find this information useful. 

113 comments:

  1. This is awesome. I've needed this so many times and didn't know it was possible. No more little scripts.

    ReplyDelete
    Replies
    1. I am glad that I saw this post. It is informative blog for us and we need this type of blog thanks for share this blog, Keep posting such instructional blogs and I am looking forward for your future posts. Python Projects for Students Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account. Project Center in Chennai

      Delete
  2. This is superb. I would possibly keep the python part in a separate script and source it in - I like to be organised! :)

    ReplyDelete
  3. Thanks you, it works ! Make sure not to set indentation to the python block, otherwise you could get some errors

    ReplyDelete
  4. Another, possibly easier way of passing variables to and from the python part is as command line arguments:

    function to_upper() {
    output=$(python - "$1" << END
    import sys
    print sys.argv[1].upper()
    END
    )
    }

    to_upper asdfasdf
    echo $output
    # gives ASDFASDF

    ReplyDelete
  5. Thanks man. Just what I was looking for.

    ReplyDelete
  6. thanks for the writeup, finally got my bash to run embedded py w arguments.

    ReplyDelete
  7. The cool thing is that you also could do it not either with python but with c/c++ or many other high language, to make thing much easier than creating sometimes nested bash-scripts

    ReplyDelete
  8. Hi and thanks for the info.

    Have a case where this doesn't seem to work as expected. I'm sure it's all me.... I'm attempting to use pssh (https://github.com/lilydjwg/pssh) which is a parallel ssh tool (sends same command to many hosts at same time).

    if you run the pssh command from cli with arguments, you'll get expected output, for example:

    pssh -h /tmp/hostlist -o /tmp/output "uname -ar"

    it'll populate /tmp/output with the output of uname -ar from all hosts listed in hostlist. perfect.


    I am trying to call this command with the same arguments from within bash but it isn't working for me.Everytime I get the command usage help, as you would if you passed nothing to it.

    Would you mind providing an example of how you would call a python script with arguments?

    thank you -

    ReplyDelete
    Replies
    1. As is often the case, I ask then figure things out shortly afterwards...

      This is what I did to resolve this, in case it helps others. Hopefully I am not spreading bad coding practices...

      I used exec..

      in the bash script:

      exec /usr/bin/python /usr/bin/pssh -h /tmp/pshhl2 -o /tmp/output "uname -ar"


      Delete
  9. Thank you for your post. This is excellent information. It is amazing and wonderful to visit your blog.
    iOS App Development Course
    iPhone App Training Course

    ReplyDelete
  10. Really good information to show through this blog. I really appreciate you for all the valuable information that you are providing us through your blog. python Online course

    ReplyDelete
  11. Hi,

    I want to pass more than one variable from bash to embedded python function and return more than one variable.

    It will be helpful if the function for this purpose can be posted for reference.

    Thanks
    Sachi

    ReplyDelete
  12. Nice blog. You have provided such a useful information in this blog. Thanks for sharing.
    DevOps Training
    DevOps Online Training

    ReplyDelete
  13. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man learn Python Online Course

    ReplyDelete
  14. Very interesting post, I get great knowledge from this post.
    thanks for shearing with us.
    GMAT online Training
    GMAT coaching in hyderabad

    ReplyDelete
  15. Thanks for sharing..
    Leanpitch provides online training in DevOps during this lockdown period everyone can use it wisely.
    DevOps Online Training

    ReplyDelete

  16. First of all i am saying that i like your post very much.I am really impressed by the way in which you presented the content and also the structure of the post. Hope you can gave us more posts like this and i really appreciate your hardwork.

    python training in bangalore

    python training in hyderabad

    python online training

    python training

    python flask training

    python flask online training

    python training in coimbatore


    ReplyDelete
  17. Thanks for sharing this informative content , Great work
    To crack scrum master interview: Scrum Master Interview Questions

    ReplyDelete
  18. Thank you so much for sharing these amazing tips. I must say you are an incredible writer, I love the way that you describe the things. Please keep sharing.
    Java Training in Chennai

    Java Training in Bangalore

    Java Training in Hyderabad

    Java Training
    Java Training in Coimbatore

    ReplyDelete
  19. Thanks for sharing this informative content , Great work
    Scrum Master Certification online

    ReplyDelete
  20. Thanks for sharing this informative content , Great work
    Leanpitch provides online training in Scrum Master during this lockdown period everyone can use it wisely.
    CSM Certification online

    ReplyDelete
  21. Thanks for sharing this informative content , Great work
    Leanpitch provides online training in Advanced Scrum Master during this lockdown period everyone can use it wisely.
    Advanced Scrum Master Training Online

    ReplyDelete
  22. Thanks for sharing all the information with us all.
    python Online Training

    ReplyDelete
  23. Thanks for sharing this informative content , Great work
    Leanpitch provides online training in Scrum Master during this lockdown period everyone can use it wisely.
    ICP-CAT certification

    ReplyDelete
  24. Thanks for sharing this informative content , Great work
    To crack scrum master interview : Scrum Master Interview Questions

    ReplyDelete
  25. Thanks for sharing this informative content , Great work
    To crack Agile coach interview : Agile coach Interview Questions

    ReplyDelete
  26. Thanks for sharing this informative content , Great work
    Leanpitch provides online training in Enterprise Agile coaching during this lockdown period everyone can use it wisely.
    Enterprise Agile Coaching

    ReplyDelete
  27. Thanks for sharing this informative content , Great work
    Leanpitch provides online training in Product Management Launchpad during this lockdown period everyone can use it wisely.
    Product Management Workshop

    ReplyDelete
  28. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    devops online training
    best devops online training
    top devops online training

    ReplyDelete
  29. Did you know that you can easily view the contents of your phone on your TV without a cable? With a screen mirror app you can easily do the screen mirroring from Android to TV. Check out www.screenmirroring.me to find out more.

    ReplyDelete
  30. This article is a great article that I have seen in my programming career so far.

    hire python developers in US

    ReplyDelete
  31. Shreeja Health Care is leading manufacturer of Oil Maker Machine. Shreeja Oil Extraction Machine is able to extract oil from various seeds like peanuts, Coconut, Sesame, Soybean, macadamia nuts, walnuts, sunflower seeds, vegetable seeds flaxseed etc.

    ReplyDelete
  32. Keytexmachines is leading CNC Machining Job Work in surat With Excellent Quality, Cost Effective Price & Prompt Delivery. CNC Machining Job Work, CNC Turning Job Work, CNC Machine Service offered by Keytex machines, Surat.

    ReplyDelete
  33. Yami immigration is a well-known and experienced immigration consultant in Surat. We provide Immigration Services for many countries such as Italy, Canada, France, Australia, Germany, USA, Malaysia, New Zealand, and Singapore.

    ReplyDelete

  34. Dr. Vivek Galani is a leading expert in skin and hair. At hair transplant clinic in Surat Skin Care, Cosmetic Laser, Hair Transplant & Slimming Center, Dr. Galani offers the most advanced cosmetic and dermatologic care treatments. The clinic uses advanced FUE methods to produce high-quality hair transplants.

    ReplyDelete
  35. This site was... how would I say it? Applicable!! At long last I have discovered something that encouraged me. Good wishes!
    live

    ReplyDelete
  36. Shreeja Health Care is leading manufacturer of Oil Making Machine. Shreeja Oil Extraction Machine is able to extract oil from various seeds like peanuts, Coconut, Sesame, Soybean, macadamia nuts, walnuts, sunflower seeds, vegetable seeds flaxseed et.

    ReplyDelete
  37. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    devops online training
    best devops online training
    top devops online training

    ReplyDelete
  38. Thanks for sharing this.,
    Leanpitch provides online training in Scrum Master during this lockdown period everyone can use it wisely.
    Join Leanpitch 2 Days CSM Certification Workshop in different cities.

    CSM certification online

    ReplyDelete
  39. Purchasing Individual Health Insurance: 3 Essential Tips From a Health Insurance Specialist By Shaun P Avery. 2000 Backlink at cheapest
    5000 Backlink at cheapest
    Boost DA upto 15+ at cheapest
    Boost DA upto 25+ at cheapest
    Boost DA upto 35+ at cheapest
    Boost DA upto 45+ at cheapest . Submitted On June 15, 2012 Suggest Article Comments Print ArticleShare this article on Facebook3Share this article on Twitter2Share

    ReplyDelete
  40. I like your blog, I read this blog please update more content on python, further check it once at
    bed correspondence
    phd regular

    ReplyDelete
  41. It was a great information and Its really worth reading it..
    Oracle Performance Tuning Course

    ReplyDelete
  42. Shreeja Health Care is leading manufacturer of Oil Maker Machine. Shreeja Oil Extraction Machine is able to extract oil from various seeds like peanuts, Coconut, Sesame, Soybean, macadamia nuts, walnuts, sunflower seeds, vegetable seeds flaxseed etc.

    ReplyDelete
  43. I read your article it is very interesting and every concept is very clear, thank you so much for sharing. AWS Certification Course in Chennai

    ReplyDelete
  44. I was really satisfied to discover this page. I needed to thank you for ones time for this awesome read!! I certainly preferred all of it and I have you saved to fav to see new data in your blog. best interiors

    ReplyDelete
  45. Brilliant post! We are connecting to this extraordinary post on our site. Keep up the extraordinary composition.
    best interiors

    ReplyDelete
  46. You should be a piece of a challenge for probably the best website on the web. I will suggest this site!
    best interiors

    ReplyDelete
  47. Ucuz, kaliteli ve organik sosyal medya hizmetleri satın almak için Ravje Medyayı tercih edebilir ve sosyal medya hesaplarını hızla büyütebilirsin. Ravje Medya ile sosyal medya hesaplarını organik ve gerçek kişiler ile geliştirebilir, kişisel ya da ticari hesapların için Ravje Medyayı tercih edebilirsin. Ravje Medya internet sitesine giriş yapmak için hemen tıkla: ravje.com

    İnstagram takipçi satın almak için Ravje Medya hizmetlerini tercih edebilir, güvenilir ve gerçek takipçilere Ravje Medya ile ulaşabilirsin. İnstagram takipçi satın almak artık Ravje Medya ile oldukça güvenilir. Hemen instagram takipçi satın almak için Ravje Medyanın ilgili sayfasını ziyaret et: instagram takipçi satın al

    Tiktok takipçi satın al istiyorsan tercihini Ravje Medya yap! Ravje Medya uzman kadrosu ve profesyonel ekibi ile sizlere Tiktok takipçi satın alma hizmetide sunmaktadır. Tiktok takipçi satın almak için hemen tıkla: tiktok takipçi satın al

    İnstagram beğeni satın almak için Ravje medya instagram beğeni satın al sayfasına giriş yap, hızlı ve kaliteli instagram beğeni satın al: instagram beğeni satın al

    Youtube izlenme satın al sayfası ile hemen youtube izlenme satın al! Ravje medya kalitesi ile hemen youtube izlenme satın almak için tıklayın: youtube izlenme satın al

    Twitter takipçi satın almak istiyorsan Ravje medya twitter takipçi satın al sayfasına tıkla, Ravje medya güvencesi ile organik twitter takipçi satın al: twitter takipçi satın al

    ReplyDelete
  48. good shit, thanks for explaining this in a way that I can actually understand ronyyguyy@gmail.com

    ReplyDelete
  49. Welcome to CapturedCurrentNews – Latest & Breaking India News 2021
    Hello Friends My Name Anthony Morris.latest and breaking news linkfeeder

    ReplyDelete
  50. Oppo launch First foldable smartphone .Oppo has teased the company’s first foldable phone.The image of the device is only shown from the spine side and we get a glimpse of the phone’s profile, revealing what appears to be a triple camera setup Article
    Click Here For website

    ReplyDelete
  51. All hosting info
    Click
    Click ther and hear
    cheap wordpress hosting
    Click ther and hear
    best vps hosting
    Click ther and hear
    best vps hosting
    Click ther and hear
    free wordpress hosting reddit
    Click ther and hear
    Click

    ReplyDelete
  52. The post is assoum. Click.https://cattlepetsdog.blogspot.com/2022/01/australian-cattle-dog-wesen.html
    https://cattlepetsdog.blogspot.com/2021/11/Australian-stumpy-tail-cattle-dog-breed-recognition-appearance.html
    https://cattlepetsdog.blogspot.com/2021/11/Australian-stumpy-tail-cattle-dog.html

    ReplyDelete
  53. Thank you for this amazing blog.
    Imperial Money is a mutual fund company in India that helps you to create wealth from your income. Imperial Money provides services like a SIP calculator, education calculator, and many more.

    ReplyDelete
  54. Step Up sip calculator or Top up calculator gives you the future value of your SIP investments which increases periodically by a given percentage.

    ReplyDelete
  55. Thanks for sharing your great idea. I am looking for this content, and suddenly I found this website. The content and your idea are super great.
    How to command f on iPad
    How to find Roku pin

    ReplyDelete
  56. Replies
    1. Imperial Money
      - Mutual Fund Investment App for You
      - Simple Design, Easy to understand
      - Made for beginners and experts both
      - Financial Planning you can do it here itself
      - Invest in the researched and ready-made basket of mutual funds recommended by experts with years together experience bringing for you.
      - Latest finance news and insights, notifications
      https://play.google.com/store/apps/details?id=com.iw.imperialmoney

      Delete
  57. Replies
    1. Top Mutual Fund Investment App
      Imperial Money
      - Mutual Fund Investment App for You
      - Simple Design, Easy to understand
      - Made for beginners and experts both
      - Financial Planning you can do it here itself
      - Invest in the researched and ready-made basket of mutual funds recommended by experts with years together experience bringing for you.
      - Latest finance news and insights, notifications
      https://play.google.com/store/apps/details?id=com.iw.imperialmoney

      Delete
  58. I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks! pm kisan status check 11 installment

    ReplyDelete

  59. A SIP calculator is a tool that allows people to calculate and estimate returns on their SIP-based mutual fund investments. SIP has recently become one of the most popular investment solutions for Generation Y (Gen-Y) also known as Millennials. Download imperial Money app and start calculating your SIP returns.sip calculator yearly
    sip calculator yearly

    ReplyDelete
  60. We SVJ Technocoat are the leading Service Provider and Exporter of an extensive array of PVD Coating Services In Surat Service and Vapor Deposition Coating Service etc. We are a well known firm for providing excellent quality coating services across the nation and in a timely manner. Owing to our improvised business models, our professionals are offering integrated solutions for our clients.

    ReplyDelete
  61. DIYAM Impex Our Company DIYAM IMPEX. We have gone from strength to strength over the years, having expanded from our core business of diamond manufacturing to Real Estate, Renewable Energy and Venture Capital. Diamond has its many utility and its industrial value is enhanced by our effective services. We now focus exclusively on CVD Diamonds Manufacturer. DIYAM IMPEX has grown to become a globally trusted and respected player in the diamond industry over the last five decades. Our expertise lies in our ability to produce a consistent supply of quality polished diamonds in all shapes and sizes.

    ReplyDelete
  62. Shreeja Oil Maker is a global leading manufacturer, wholesaler, supplier, exporter of various small scale machinery. Shreeja oil maker machine or Mini oil extraction machine is one of our innovative product which is able to extract 100% pure oil from the various seed. This is also known as a Oil Extraction Machine or mini oil Ghani. We have a stronghold in national as well as a worldwide market.

    ReplyDelete
  63. Candela GentleLASE medical grade laser applies precise, controlled pulses of laser. Laser Hair Removal Treatment in Auckland energy that reach down the hair shaft into the follicle underneath the skin, cauterizing the hair at its root. the encompassing tissue or skin isn’t damaged. The laser’s gentle beam of sunshine damages and consequently prevents the follicle from growing.

    ReplyDelete
  64. FAB VOGUE STUDIO provide all kind of Fabric which can be use for Making any designer apparel. kota doria fabric We guarantee you’ll love working with us. Design Customization Available. Wholesale price available on request. Quality Assurance. COD Available.

    ReplyDelete
  65. RQC is one of the Best Surat Hair Transplant Clinic, with services, including Hair Transplant, Hair Treatment, Hairfall and Hair Loss, dermatology. RQC brings you the best services in hair transplant, Hair Treatment.

    ReplyDelete
  66. İnstagram takipçi satın al! İnstagram takipçi sitesi ile takipçi satın al sende sosyal medyada fenomen olmaya bir adım at. Sende hemen instagram takipçi satın almak istiyorsan tıkla:

    1- takipçi satın al

    2- takipçi satın al

    3- takipçi satın al

    ReplyDelete
  67. It is a more helpful and time saving post. What is a Turkish visa online ? It is nothing more than a substitute for a Turkish physical visa. but the application process is more simple than the Turkish physical visa. You can also check the details about the Turkish online visa.


    ReplyDelete
  68. Replies
    1. Invest in equity mutual funds - small cap, large cap, mid cap, multi-cap - for the long term and higher returns. Check out SIP Calculator to know how much returns you can make.
      Safe & Secure We use the latest security standards to keep your data safe and encrypted.

      Delete
  69. This comment has been removed by the author.

    ReplyDelete
  70. Start Your SIP in Just a Few Clicks with the Imperial Money app, Where the Team of Expert take care of your Portfolio & has 24/7 customer Support Download Now-https://play.google.com/store/apps/details?id=com.iw.imperialmoney

    DOWNLOAD NOW

    ReplyDelete
  71. I love this text for the skillfully explored content and eminent phrasing. I got for that excuse rehearsing in this fabric that I couldnt decline considering. i'm intrigued once your satisfaction out and introduce. much thanks to you accurately a reasonable plan. thanks! Edius Pro 9 Download Full Version With Crack

    ReplyDelete
  72. really pleasurable state. I just staggered a propos your weblog and wanted to guarantee that I have totally cherished perusing your blog entries. Any eccentricity I will buy in your feed and that I want you articulate close to more rapidly. huge gratitude for the helpful data. thanks! Bitdefender Total Security Crack

    ReplyDelete
  73. IMPERIAL MONEY APP - KEY FEATURES
    Feature for Convenience -
    Start, stop or increase SIP online anytime -
    KYC process can be completed within the app through video KYC -
    Simple and elegant design for stress-free investing -
    Queries can be raised through tickets at the support desk anytime -
    Free of cost account opening, transactions, and recommendations -
    No hidden charges Download Now

    ReplyDelete
  74. IMPERIAL MONEY SERVICES
    # We are certified financial planners and we focus on providing our clients
    # with investment plans as per their financial goals
    # Our success is dependent on yours We do well when you are doing well.
    # This is our motivation to make sure your finances are in good shape.

    IMPERIAL MONEY APP DOWNLOAD NOW

    ReplyDelete

  75. What are the Benefits of Investing in Mutual Fund?
    ● Professional Management
    ● Asset Allocation
    ● Best Tax Saving Option
    ● Schemes for Every Financial Goal
    ● High Return on investments
    ● Easiest Form Of Investment
    ● Safety & Transparency
    ● Liquidity
    To Know More
    IMPERIAL MONEY APP DOWNLOAD NOW

    ReplyDelete