Page 1 of 1

Website Schedule scrape ($)

PostPosted: Mon Feb 22, 2016 3:16 pm
by exos
Hello,

In my place of work, my schedule for the next day is posted sometimes between 2pm-7pm. I am looking for someone to help set up a web scrape to:

1) Determine if the schedule has been released for the following day.

and hopefully...

2) Pull any pertinent data for my self (by name search) and either set into google calendar or in an indigo variable.


If anyone is interested please let me know! I do check my private messages on here frequently and we can chat.

Re: Website Schedule scrape ($)

PostPosted: Wed Feb 24, 2016 2:57 pm
by wiery
a simple bash script with lynx / curl should give all the info you need. Failing that I used to use iMacros via shell script that launched firefox, got my broadband usage and dumped it to a file, from there you can grep the information you are looking for and insert into Indigo.

Have you an example page you can save and upload here ?

Re: Website Schedule scrape ($)

PostPosted: Wed Feb 24, 2016 3:00 pm
by exos
Better yet... I'll send you directly to the scheduling page. As you see, it'll populate with the current date. Currently I need to random check the site, click the following day's date, enter my name in the search, and see if anything comes back.

https://www.cnatra.navy.mil/scheds/sche ... px?sq=vt-9

Re: Website Schedule scrape ($)

PostPosted: Wed Feb 24, 2016 3:15 pm
by wiery
That sounds exactly like a job for iMacro. I installed firefox on my Mac for this sole purpose. You record the actions and then have the shell script run the macro. I have it on the Mac at home so will gather details for you, just away at the minute but i'd recommend installing Firefox, set the home page to the URL and record a macro of your actions. You can then save the output page to a file and we can use bash to extract the details.

Re: Website Schedule scrape ($)

PostPosted: Wed Feb 24, 2016 3:52 pm
by exos
Sounds great let me set it up