Website Schedule scrape ($)

Posted on
Mon Feb 22, 2016 3:16 pm
exos offline
Posts: 57
Joined: Feb 02, 2016

Website Schedule scrape ($)

Hello,

In my place of work, my schedule for the next day is posted sometimes between 2pm-7pm. I am looking for someone to help set up a web scrape to:

1) Determine if the schedule has been released for the following day.

and hopefully...

2) Pull any pertinent data for my self (by name search) and either set into google calendar or in an indigo variable.


If anyone is interested please let me know! I do check my private messages on here frequently and we can chat.

Posted on
Wed Feb 24, 2016 2:57 pm
wiery offline
Posts: 288
Joined: Jun 02, 2008
Location: Ireland

Re: Website Schedule scrape ($)

a simple bash script with lynx / curl should give all the info you need. Failing that I used to use iMacros via shell script that launched firefox, got my broadband usage and dumped it to a file, from there you can grep the information you are looking for and insert into Indigo.

Have you an example page you can save and upload here ?

Posted on
Wed Feb 24, 2016 3:00 pm
exos offline
Posts: 57
Joined: Feb 02, 2016

Re: Website Schedule scrape ($)

Better yet... I'll send you directly to the scheduling page. As you see, it'll populate with the current date. Currently I need to random check the site, click the following day's date, enter my name in the search, and see if anything comes back.

https://www.cnatra.navy.mil/scheds/sche ... px?sq=vt-9

Posted on
Wed Feb 24, 2016 3:15 pm
wiery offline
Posts: 288
Joined: Jun 02, 2008
Location: Ireland

Re: Website Schedule scrape ($)

That sounds exactly like a job for iMacro. I installed firefox on my Mac for this sole purpose. You record the actions and then have the shell script run the macro. I have it on the Mac at home so will gather details for you, just away at the minute but i'd recommend installing Firefox, set the home page to the URL and record a macro of your actions. You can then save the output page to a file and we can use bash to extract the details.

Posted on
Wed Feb 24, 2016 3:52 pm
exos offline
Posts: 57
Joined: Feb 02, 2016

Re: Website Schedule scrape ($)

Sounds great let me set it up

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 2 guests