Reply to: php question get remote url

[xoggoth]

Cheers for further comments Nick. Turns out there was a security problem, I have to request access via the host firewall for every URL I want to open.

[NickFitz]

You shouldn't use htmlentities(), as it's not relevant and could damage your URLs. See my post at the end of page 1, and the php.net documentation I linked to in my previous post - explicitly opening, reading, and closing the file may be more reliable for your setup.

Alternatively try the cURL functions - they're slightly more complicated to use, but if nothing else they give you a much better indication of what, if anything, has gone wrong.

[xoggoth]

Hey lotsa of sensible answers, cheers. Will try those later, if work then on to the hard bit, actually extracting the figure I want.

Know nothing about the security stuff or server side at all as is probably obvious but this is a short conversion file Worlpay provide configured to my specific account so must be safe enough to use it?



****
Hmm! HTMLentities works but now I am getting a file timeout although pages are available, same if use fopen. All the things that should allow remote access like allow_url_fopen are set to on according to the info file. Bum.

[NickFitz]

you got me there

I bow down to NickFitz's superior knowledge

As so must we all

(otherwise he'll never shut up)

[bogeyman]

you got me there

I bow down to NickFitz's superior knowledge

As so must we all

(otherwise he'll never shut up)

[Ardesco]

you got me there

I bow down to NickFitz's superior knowledge

[NickFitz]

Yup, it is. He needs to use htmlentities() instead (see my original post). urlencode() will convert all the symbols to their html equivalent which will break a file_get_contents callbecause it doesn't realise it is a url any more.

Nope - he's building a URL, which isn't HTML. htmlentities() is used for encoding stuff to be rendered on a web page, not for URL encoding; so if one writes a URL to a page, it should be formed with urlencode() applied to values in the query string and fragment identifier, and then passed through htmlentities() to make it valid HTML. But for use as a URL, only the first is appropriate, otherwise it'll mess up the query string.

E.G. http://example.com?p=1&q=2 would become http://example.com?p=1&q=2, which would result in the server thinking it had been passed a value of 2 with the name amp;q.

[NickFitz]

Unless of course your friendly web admin has set:

disable_functions = phpinfo 

in the php.ini

Misuse of the word "friendly" there - well, unless they allow me to re-enable it when I want to

[Ardesco]

Actually xog, your problem is almost certainly the urlencode call: this turns the slashes and so on into their urlencoded form.

Assuming xxxxx represents some value that might contain special characters and needs URLEncoding, try

$encoded_inst_id = urlencode($inst_id);
$contents = file_get_contents('https://select.worldpay.com/wcc/info?op=rates&instId='.$encoded_inst_id);
echo $contents;

Or have a look at the first example at http://www.php.net/manual/en/features.remote-files.php if you want finer control over file processing.

Yup, it is. He needs to use htmlentities() instead (see my original post). urlencode() will convert all the symbols to their html equivalent which will break a file_get_contents call because it doesn't realise it is a url any more.

[NickFitz]

I know that you silly bonobo! It should not, however, be enabled by default.

Why not? It doesn't do any harm. If somebody leaves a security hole in their application that allows it to be abused, then they're almost certainly leaving SQL injection and XSS holes as well, so this is the least of their worries.

[NickFitz]

Still trying to sort out a currency conversion on my site and having a bash at some php:-

$contents = file_get_contents(urlencode('https://select.worldpay.com/wcc/info?op=rates&instId=xxxxx'));
echo $contents;

This gives me:

Warning: file_get_contents(blah blah) [function.file-get-contents]: failed to open stream: No such file or directory in (my host directory)/blahblah.php

Why is it looking for the file locally? Get same with other urls like www.google.com but as I understand it this should work with a remote URL if echo ini_get("allow_url_fopen"); returns 1 which it does.

Cheers

Actually xog, your problem is almost certainly the urlencode call: this turns the slashes and so on into their urlencoded form.

Assuming xxxxx represents some value that might contain special characters and needs URLEncoding, try

$encoded_inst_id = urlencode($inst_id);
$contents = file_get_contents('https://select.worldpay.com/wcc/info?op=rates&instId='.$encoded_inst_id);
echo $contents;

Or have a look at the first example at http://www.php.net/manual/en/features.remote-files.php if you want finer control over file processing.

[Ardesco]

How do you expect a server-side script to retrieve data from another server if it can't open a URL? This is one of the fundamental aspects of the way the web works. It doesn't exist solely for the purpose of providing HTML pages to browsers.

As far as I can see the only possible reason for a hosting provider to disable this would be over bandwidth concerns. I wouldn't use a hosting provider that didn't allow this.

xoggoth: you might have more success using the cURL functions. For retrieving an https URL, your hosting provider will have to have included OpenSSL support (or some equivalent library) in their PHP installation (which they should have done if they're any good).

You can check by uploading a temporary file containing the following:

<?php
phpinfo()
?>

as, say, info.php and going to that page. Scroll down to the "Configuration" section after the copyright notices (about one screen down). The "PHP Core" section should include a row for allow_url_fopen which should have the "Local value" of "on". Scrolling further, you should have subsections for "curl", meaning cURL functions are installed, and "openssl", meaning you can use https URLs - for both of those the first row should state that support is "enabled", otherwise you're out of luck.

Oh, and then remove the info.php file, as it provides information about your server that the rest of the world shouldn't be allowed to see

Unless of course your friendly web admin has set:

disable_functions = phpinfo 

in the php.ini

[bogeyman]

How do you expect a server-side script to retrieve data from another server if it can't open a URL? This is one of the fundamental aspects of the way the web works. It doesn't exist solely for the purpose of providing HTML pages to browsers.

I know that you silly bonobo! It should not, however, be enabled by default.

[Ardesco]

Depends which PHP distro you get I would assume, but the default php.ini you get from php.net has it on by default.

At the end of the day the webserver admin should know about it and decide if it should be on or off, as there are many legitimate uses for it. Its the so called web admins who don't check the default php.ini who are a bag 'o tulipe, not PHP itself.

[NickFitz]

Correct.

But is that still the php.ini default? Hope not

PHP! Bag 'o tulipe!

How do you expect a server-side script to retrieve data from another server if it can't open a URL? This is one of the fundamental aspects of the way the web works. It doesn't exist solely for the purpose of providing HTML pages to browsers.

As far as I can see the only possible reason for a hosting provider to disable this would be over bandwidth concerns. I wouldn't use a hosting provider that didn't allow this.

xoggoth: you might have more success using the cURL functions. For retrieving an https URL, your hosting provider will have to have included OpenSSL support (or some equivalent library) in their PHP installation (which they should have done if they're any good).

You can check by uploading a temporary file containing the following:

<?php
phpinfo()
?>

as, say, info.php and going to that page. Scroll down to the "Configuration" section after the copyright notices (about one screen down). The "PHP Core" section should include a row for allow_url_fopen which should have the "Local value" of "on". Scrolling further, you should have subsections for "curl", meaning cURL functions are installed, and "openssl", meaning you can use https URLs - for both of those the first row should state that support is "enabled", otherwise you're out of luck.

Oh, and then remove the info.php file, as it provides information about your server that the rest of the world shouldn't be allowed to see