Spiders

[cmhardwick]

I was reloading and reconfiguring my xlobby (drive crash). I put the spiders from the zip files in the spiders directory and in the intheaters text file, I changed the variable from a 0 to a 1 but I sitll just get the low-res images. If i go into the database and choose the spider and try to run it from there it just flashes and says done, nothing retrieved. Granted, I may have screwed something up hehehe I've been known to do that, but I don't think so.

Thanks TSWhite!

[tswhite70]

The impaawards.com spider from the UpdatedDVDspiders.zip (12/3/06) is still working, I just downloaded it and retested just to make sure.

As for Intheaters - I just downloaded v2.5 again and checked it, it's still getting HQ coverart from impaawards. You did set HQ=1 right?

Can you double check the version of the spiders and the version of Intheaters you are running, old versions is the only explanation I can think of.

good luck,
tsw

[cmhardwick]

Ok, I started over again and reinstalled the xscriptng plugin. Seems to be working now. I've still got one movie I don't have art for, but I can manually get the odd movie here and there.

Thanks again. I don't know why it wasn't working, but it's working now!

[tswhite70]

cmhardwick - You were right about impawards.com (actually google I think), they did change. I tried the spider again tonight and it wasn't working so I've updated for the change. Hopefully they'll leave it alone for a while.

Also included are new versions of Walmart.com and IMDB.com (added large coverart) + a new IMDB.com-Direct version (for those movies that go directly to the movie page - see this thread: http://www.xlobby.com/forum/viewtopic.php?p=36183). Amazon.com & Buy.com spiders from 12/3/06 are included.

http://home.comcast.net/~twhite644/spid ... piders.zip

good luck,
tsw

[jaydwil]

Has anyone noticed that the info for IMDB spiders has stopped working again? Perhaps they've changed their site layout. I've got the latest version posted to the boards from 1/16/07. Are there any other versions floating around?

[lar282]

try this
//----------------------------------------------------------------
url=http://us.imdb.com/find?s=all&q=%searchstring%;tt=1
results=<a href="(?<url>/title/.*?/).*?">(?<display>.*?)</a>

// Now goto that new url that stores all info on imdb
lasse=http://www.imdb.com%url%

<title>(?<display>.*?)\((?<year>.*?)\)</title>
//<a name="poster" .*? title="(?<title>.*?)">

//Plot
<b class="ch">Plot Outline:</b> (?<plot>.*?)?. <a href="

// Coverart
<a name="poster" .*? title=".*?" src="(?<coverart>.*?)"
//<a name="poster" href=".*?" title=.*><img border="0" alt=.*? title=.*? src="(?<coverart>http://ia.ec.imdb.com/.*?)".*?></a>
//replace=coverart:m.jpg:f.jpg


<a href="/Sections/Genres/.*?">(?<genre>.*?)</a>

<a href="/mpaa">MPAA</a>:</b>(?<rating>.*?)<br>

<b class="ch">Runtime:</b>(?<runtime>.*?)<br>

//Actors
// <table cellpadding="1" cellspacing="0"><tr><td colspan="4" align="left"><b class="blackcatheader">Cast overview, first billed only:
first billed only: </b></td></tr> (?<variable><tr>.*?</tr><tr>.*?</tr><tr>.*?</tr><tr>.*?</tr><tr>.*?</tr>)
<td valign="top"><a href="/name/.*?">(?<actors>.*?)</a></td>

//----------------------------------------------------------------

[jaydwil]

Thanks for the reply. The only thing that comes up when I use your code (after deleting the trailing spaces) is the info below when I search for Babel.

display: Babel
year: 2006
genre: Drama,Thriller

Looking at the file, I would assume I should be seeing the Rating, Runtime, Actors and Plot as well. Do those work for you?

[lar282]

hmmm, the prob did change it.I'll look at it tomorow

//Lasse

[lar282]

I can't get it to work...
this works as a regular expression but not in Xlobby.Do not know why.Maybe somebody else knows?

(?<plot>.*?) <a class="tn15more inline" href="/rg/title-tease/plotsummary/title/*.


test url is
http://www.imdb.com/title/tt0449467/

and the regex page is:
http://regexlib.com/RETester.aspx

//Lasse

[tswhite70]

lar282 - I don't know if you mis-copied your regex, but for plot shouldn't it be:

Short plot on main page:

Code: Select all

<h5>Plot Outline:</h5>.*?\r\n(?<plot>.*?) <a class=

Full plot from "more" link:

Code: Select all

<h5>Plot Outline:</h5>.*?<a class="tn15more inline" href="(?<url>/rg/title-tease/plotsummary/title/.*?/plotsummary)">more</a>
<p class="plotpar">.*?\r\n(?<plot>.*?)\r\n<i>

IMDB definitely changed, see here: http://us.imdb.com/help/show_leaf?redesignfaq

I whipped up a new version of the spider, but I won't be able to test it till tonight. Once I get it working I'll add it to my UpdateDVDSpiders zip file...

New version (untested...)

Code: Select all

url=http://us.imdb.com/find?q=%searchstring%;tt=1
results=<a href="(?<url>/title/.*?/).*?">(?<display>.*?)</a>

//title & year
<div id="tn15title">.*?<h1>(?<title>.*?) <span>\(<a href="/Sections/Years/.*?">(?<year>.*?)</a>\)</span></h1>

//Directors
<h5>Directed by</h5>(?<variable>.*?)<br/>
<a href="/name/.*?">(?<director>.*?)</a>

//Genre
<a href="/Sections/Genres/.*?">(?<genre>.*?)</a>

//Rating
<h5><a href="/mpaa">MPAA</a>:</h5>.*?\r\n(?<rating>.*?)\r\n</div>

//Runtime
<h5>Runtime:</h5>.*?\r\n(?<runtime>.*?)\s\r\n</div>

//Actors
\(Cast overview, first billed only\)(?<variable>.*?)<br/>
<a href="/name/.*?">(?<actors>.*?)</a></td>


//Plot
<h5>Plot Outline:</h5>.*?<a class="tn15more inline" href="(?<url>/rg/title-tease/plotsummary/title/.*?/plotsummary)">more</a>
<p class="plotpar">.*?\r\n(?<plot>.*?)\r\n<i>

//Coverart
<a name="poster" href="photogallery" title=".*?"><img border="0" alt=".*?" title=".*?" src="(?<coverart>.*?)".*?></a>

replace=coverart:m.jpg:f.jpg

good luck,
tsw

[lar282]

the source looks this this:
-------------------------------------------
<div class="info">
<h5>Plot Outline:</h5>
Tragedy strikes a married couple on vacation in the Moroccan desert, touching off an interlocking story involving four different families. <a class="tn15more inline" href="/rg/title-tease/plotsummary/title/tt0449467/plotsummary">more</a>
</div>

----------------------------------
so the
<h5>Plot Outline:</h5>.*?\r\n(?<plot>.*?) <a class=
wont work on my system, but maybe I am doing something wrong.

//Lasse

[lar282]

tried your new spider.It works but not for the plot. here is output from a search of babel

title: Babel
year: 2006
director: Alejandro Gonz�lez I��rritu
genre: Drama,Thriller
actors: Brad Pitt,Cate Blanchett,Mohamed Akhzam,Peter Wight,Harriet Walter,Trevor Martin,Matyelok Gibbs,Georges Bousquet,Claudine Acs,Andr� Oumansky,Michael Maloney,Dermot Crowley,Wendy Nottingham,Henry Maratray,Linda Broughton

[tswhite70]

Updated the zip file, should be working now. (fyi, my \r\n code for end of line was the problem, I guess Xlooby doesn't see the line breaks in the source the same way we do when I view it in notepad...)

http://home.comcast.net/~twhite644/spid ... piders.zip

good luck,
tsw

[lar282]

Thank u tsw



//Lasse

[jaydwil]

Yep, works great! Thanks a lot!!!