包简介

PHP class that attempts to resolve URLs to a final, canonical link.

README 文档

README

URLResolver.php is a PHP class that attempts to resolve URLs to a final, canonical link. On the web today, link shorteners, tracking codes and more can result in many different links that ultimately point to the same resource. By following HTTP redirects and parsing web pages for open graph and canonical URLs, URLResolver.php attempts to solve this issue.

Patterns Recognized

Follows 301, 302, and 303 redirects found in HTTP headers
Follows Open Graph URL <meta> tags found in web page <head>
Follows Canonical URL <link> tags found in web page <head>
Aborts download quickly if content type is not an HTML page

I am open to additional suggestions for improvement.

Usage

Resolving a URL can be as easy as:

<?php require_once('URLResolver.php');

$resolver = new mattwright\URLResolver();
print $resolver->resolveURL('http://goo.gl/0GMP1')->getURL();

If you installed this library using composer, you would change the first line above to:

<?php require_once('vendor/autoload.php');

However, in most cases you will want to perform a little extra setup. The following code sets a user agent to identify your crawler (otherwise the default will be used) and also designates a temporary file that can be used for storing cookies during the session. Some web sites will test the browser for cookie support, so this will enhance your results.

<?php require_once('URLResolver.php');
$resolver = new mattwright\URLResolver();

# Identify your crawler (otherwise the default will be used)
$resolver->setUserAgent('Mozilla/5.0 (compatible; YourAppName/1.0; +http://www.example.com)');

# Designate a temporary file that will store cookies during the session.
# Some web sites test the browser for cookie support, so this enhances results.
$resolver->setCookieJar('/tmp/url_resolver.cookies');

# resolveURL() returns an object that allows for additional information.
$url = 'http://goo.gl/0GMP1';
$url_result = $resolver->resolveURL($url);

# Test to see if any error occurred while resolving the URL:
if ($url_result->didErrorOccur()) {
	print "there was an error resolving $url:\n  ";
	print $url_result->getErrorMessageString();
}

# Otherwise, print out the resolved URL.  The [HTTP status code] will tell you
# additional information about the success/failure. For instance, if the
# link resulted in a 404 Not Found error, it would print '404: http://...'
# The successful status code is 200.
else {
	print $url_result->getHTTPStatusCode();
	print ': ';
	print $url_result->getURL();
}

Installation and Requirements

License

URLResolver.php is licensed under the MIT License, viewable in the source code.

Install with Composer

composer require mattwright/urlresolver

Download

URLResolver.php as a .tar.gz or .zip file.

Requirements

The curl extension must be installed as part of PHP
PHP Simple HTML DOM Parser is required and included with the download.

API

URLResolver()

$resolver = new mattwright\URLResolver();
Create the URL resolver object that you call additional methods on.

$resolver->resolveURL($url);
$url is the link you want to resolve.
Returns a [URLResult] object that contains the final, resolved URL.

$resolver->setUserAgent($user_agent);
Pass in a string that is sent to each web server to identify your crawler.

$resolver->setCookieJar($cookie_file); # Defaults to disable cookies
*** This file will be removed at the end of each resolveURL() call. ***
Pass in the path to a file used to store cookies during each resolveURL() call.
If no cookie file is set, cookies will be disabled and results may suffer.
This file must not already exist. If it does, pass true as second argument to enable overwrite.

$resolver->setMaxRedirects($max_redirects); # Defaults to 10
Set the maximum number of URL requests to attempt during each resolveURL() call.

$resolver->setMaxResponseDataSize($max_bytes); # Defaults to 120000
Pass in an integer specifying the maximum data to download per request.
Multiple URL requests may occur during each resolveURL() call.
Setting this too low may limit the usefulness of results (default 120000).

$resolver->setRequestTimeout($num_seconds); # Defaults to 30
Set the maximum amount of time, in seconds, any URL request can take.
Multiple URL requests may occur during each resolveURL() call.

$resolver->setPreferCanonicalURL($value); # Defaults to false
Set $value to true to prioritize canonical URL over Open Graph URL.

$resolver->isDebugMode($value); # Defaults to false
Set $value to true to enable debug mode and false to disable (the default).
This will print out each link visited, along with status codes and link types.

URLResolverResult()

$url_result = $resolver->resolveURL($url);
Retrieve the URLResolverResult() object representing the resolution of $url.

$url_result->getURL();
This is the best resolved URL we could obtain after following redirects.

$url_result->getHTTPStatusCode();
Returns the integer HTTP status code for the resolved URL.
Examples: 200 - OK (success), 404 - Not Found, 301 - Moved Permanently, ...

$url_result->hasSuccessHTTPStatus();
Returns true if the HTTP status code for the resolved URL is 200.

$url_result->hasRedirectHTTPStatus();
Returns true if the HTTP status code for the resolved URL is 301, 302, or 303.

$url_result->getContentType();
Returns the value of the Content-Type HTTP header for the resolved URL.
If header not provided, null is returned. Examples: text/html, image/jpeg, ...

$url_result->getContentLength();
Returns the size of the fetched URL in bytes for the resolved URL.
Determined only by the Content-Length HTTP header. null returned otherwise.

$url_result->isOpenGraphURL();
Returns true if resolved URL was marked as the Open Graph URL (og:url)

$url_result->isCanonicalURL();
Returns true if resolved URL was marked as the Canonical URL (rel=canonical)

$url_result->isStartingURL();
Returns true if resolved URL was also the URL you passed to resolveURL().

$url_result->didErrorOccur();
Returns true if an error occurred while resolving the URL.
If this returns false, $url_result is guaranteed to have a status code.

$url_result->getErrorMessageString();
Returns an explanation of what went wrong if didErrorOccur() returns true.

$url_result->didConnectionFail();
Returns true if there was a connection error (no header or no body returned).
May indicate a situation where you are more likely to try at least once more.
If this returns true, didErrorOccur() will true as well.

Changelog

v2.0 - January 17, 2019
- Breaking change: namespaced the library for use with composer psr-4
- Add requested option to prefer canonical URL over Open Graph
- Minor fixes / improvements
- Upgrade simple_html_dom to 1.8.1
v1.1 - June 3, 2014
- Support http redirect code 303
v1.0 - December 3, 2011
- Initial release supports http header redirects, og:url and rel=canonical

mattwright/urlresolver 适用场景与选型建议

mattwright/urlresolver 是一款基于 PHP 开发的 Composer 扩展包，目前已累计 173.21k 次下载、GitHub Stars 达 66，最近一次更新时间为 2019 年 01 月 18 日，在 PHP 生态内属于活跃度较高的组件。

它主要适用于以下技术方向：「url」「redirect」「link」「resolve」「canonical」等业务场景。在实际项目中，围绕这些方向常见需要落地的问题包括：接口对接、性能调优、并发安全、与既有框架（Laravel / ThinkPHP / Yii / Webman 等）的兼容适配，以及生产环境的日志埋点与稳定性保障。

我们在过去多个企业项目中使用过 mattwright/urlresolver 或与其功能相近的方案，如果你在选型或落地过程中遇到问题，例如 版本兼容、二次改造、私有化封装、与内部系统对接、生产 BUG 排查，欢迎联系我们协助评估。

围绕 mattwright/urlresolver 我们能提供哪些服务？

定制开发 / 二次开发

基于 mattwright/urlresolver 在你已有业务上做功能扩展、字段裁剪、UI 适配、与内部账号 / 权限 / 日志系统的深度对接。

BUG 修复 & 性能优化

线上偶发问题、内存泄漏、慢查询、并发异常等排查修复；针对高流量场景做缓存、队列、索引层面的调优。

项目外包 & 长期维护

承接完整的项目从需求 → 设计 → 开发 → 上线 → 长期运维；也可按月提供技术保姆服务。

yvsm@zunyunkeji.com QQ：316430983 微信：yvsm316 西安尊云信息科技 · 专注 PHP / Go / 分布式系统研发

与 mattwright/urlresolver 相关的其它包

同方向 / 同关键字的高下载量 PHP Composer 包推荐，方便对比选型：

dynamic/silverstripe-linkable 1

A couple of handy form fields and objects for managing external and internal links on DataObjects

in2code/ipandlanguageredirect 14

Redirects TYPO3 visitors automatic or with a suggestlink to another language and/or root page.

antevenio/pdo-mysql-query-linker 0

PHP library that allows linking queries from diferent physical databases using mysql pdo database connections

ericlagarda/novalinkresource 17

Custom links on your sidebar for Laravel Nova.

cmsexperts/link2language 17

Set Links with a specific language parameter

ruthgeridema/laravel-url-rewrites 29

Easy URL rewrites in your Laravel application

mattwright/urlresolver

包简介

关键字：