迁移geonode基url

这个 migrate_baseurl Management Command 允许您在出于某种原因需要更改 Domain Name 属于 IP Address 地理节点的。

这个 must 也可用于需要从 HTTPHTTPS ,作为实例。

首先让我们看看 --help 选择权 migrate_baseurl 管理命令,以检查所有命令选项和功能。

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --help

注解

如果启用 local_settings.py 命令将更改如下:

DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py migrate_baseurl --help

这将产生如下输出

usage: manage.py migrate_baseurl [-h] [--version] [-v {0,1,2,3}]
                             [--settings SETTINGS]
                             [--pythonpath PYTHONPATH] [--traceback]
                             [--no-color] [-f]
                             [--source-address SOURCE_ADDRESS]
                             [--target-address TARGET_ADDRESS]

Migrate GeoNode VM Base URL

optional arguments:
-h, --help            show this help message and exit
--version             show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
--settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
--pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
--traceback           Raise on CommandError exceptions
--no-color            Don't colorize the command output.
-f, --force           Forces the execution without asking for confirmation.
--source-address SOURCE_ADDRESS
                        Source Address (the one currently on DB e.g.
                        http://192.168.1.23)
--target-address TARGET_ADDRESS
                        Target Address (the one to be changed e.g. http://my-
                        public.geonode.org)
  • 例1 :我想将GeoNode实例从 http:\\127.0.0.1http:\\example.org

    警告

    始终确保您正在使用 对的 设置

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=127.0.0.1 --target-address=example.org
    
  • 例2 :我想将GeoNode实例从 http:\\example.orghttps:\\example.org

    警告

    始终确保您正在使用 对的 设置

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=http:\\example.org --target-address=https:\\example.org
    
  • 例3 :我想将GeoNode实例从 https:\\example.orghttps:\\geonode.example.org

    警告

    始终确保您正在使用 对的 设置

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=example.org --target-address=geonode.example.org
    

注解

迁移基本URL之后,请确保还清理链接和目录元数据 (更新权限、元数据、图例和下载链接

将数据加载到geonode

在某些情况下,不可能或不方便使用 Upload Form 通过web界面向geonode添加新层。例如:

  • 数据集太大,无法通过web界面上传。

  • 我们希望以编程方式从大容量存储导入一些数据。

  • 我们想从数据库中导入一些表。

  • 我们需要先处理数据,然后,也许,把它转换成另一种格式。

本节将介绍从geoserver、命令行或以编程方式将数据加载到geonode的各种可用选项。

警告

本节的某些部分摘自 GeoServer 项目和培训文件。

管理指挥部 importlayers

这个 geonode.layers django应用程序包含两个管理命令,您可以使用它们在geonode中加载或配置数据。

它们都可以通过使用 manage.py 脚本。

首先让我们看看 --help 选择权 importlayers 管理命令,以检查所有命令选项和功能。

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers --help

注解

如果启用 local_settings.py 命令将更改如下:

DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py importlayers --help

这将产生如下输出

usage: manage.py importlayers [-h] [--version] [-v {0,1,2,3}]
                            [--settings SETTINGS] [--pythonpath PYTHONPATH]
                            [--traceback] [--no-color] [-u USER] [-i] [-o]
                            [-k KEYWORDS] [-l LICENSE] [-c CATEGORY]
                            [-r REGIONS] [-n LAYERNAME] [-t TITLE]
                            [-a ABSTRACT] [-d DATE] [-p] [-m] [-C CHARSET]
                            [path [path ...]]

Brings a data file or a directory full of data files into a GeoNode site.
Layers are added to the Django database, the GeoServer configuration, and the
pycsw metadata index.

positional arguments:
path                  path [path...]

optional arguments:
-h, --help            show this help message and exit
--version             show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
--settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
--pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
--traceback           Raise on CommandError exceptions
--no-color            Don't colorize the command output.
-u USER, --user USER  Name of the user account which should own the imported
                        layers
-i, --ignore-errors   Stop after any errors are encountered.
-o, --overwrite       Overwrite existing layers if discovered (defaults
                        False)
-k KEYWORDS, --keywords KEYWORDS
                        The default keywords, separated by comma, for the
                        imported layer(s). Will be the same for all imported
                        layers if multiple imports are done in one command
-l LICENSE, --license LICENSE
                        The license for the imported layer(s). Will be the
                        same for all imported layers if multiple imports are
                        done in one command
-c CATEGORY, --category CATEGORY
                        The category for the imported layer(s). Will be the
                        same for all imported layers if multiple imports are
                        done in one command
-r REGIONS, --regions REGIONS
                        The default regions, separated by comma, for the
                        imported layer(s). Will be the same for all imported
                        layers if multiple imports are done in one command
-n LAYERNAME, --name LAYERNAME
                        The name for the imported layer(s). Can not be used
                        with multiple imports
-t TITLE, --title TITLE
                        The title for the imported layer(s). Will be the same
                        for all imported layers if multiple imports are done
                        in one command
-a ABSTRACT, --abstract ABSTRACT
                        The abstract for the imported layer(s). Will be the
                        same for all imported layers if multiple imports are
                        done in one command
-d DATE, --date DATE  The date and time for the imported layer(s). Will be
                        the same for all imported layers if multiple imports
                        are done in one command. Use quotes to specify both
                        the date and time in the format 'YYYY-MM-DD HH:MM:SS'.
-p, --private         Make layer viewable only to owner
-m, --metadata_uploaded_preserve
                        Force metadata XML to be preserved
-C CHARSET, --charset CHARSET
                        Specify the charset of the data

虽然对大多数选项的描述应该是不言而喻的,但值得更详细地回顾一些关键选项。

  • 这个 -i 选项将强制命令在第一次遇到错误时停止。如果未指定此选项,进程将跳过具有层的错误并继续加载其他层。

  • 这个 -o 选项指定与基名同名的层将被加载并覆盖现有层。

  • 这个 -u 选项指定哪个用户将拥有导入的图层。同一个用户将是该层的联系人和元数据作者

  • 这个 -k 选项用于为导入的所有图层添加关键字。

  • 这个 -C 选项指定数据的字符编码。

通过如上所述指定选项并指定单个图层文件或包含多个文件的目录的路径,可以调用“导入图层管理”命令。出于本练习的目的,让我们使用geonode附带的默认测试层集。您可以将此路径替换为指向您自己形状文件的目录。

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers -v 3 /var/lib/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/

此命令将生成以下输出到您的终端

Verifying that GeoNode is running ...
Found 8 potential layers.
No handlers could be found for logger "pycsw"
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp' (1/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_coastline.shp' (2/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_highway.shp' (3/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_location.shp' (4/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_natural.shp' (5/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_poi.shp' (6/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_water.shp' (7/8)
[created] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/single_point.shp' (8/8)

Detailed report of failures:

Finished processing 8 layers in 30.0 seconds.

8 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
3.750000 seconds per layer

如果在运行此命令时遇到错误,可以使用 -v 选项以增加输出的详细程度,以便可以调试问题。

详细级别可以从 0-3 具有 0 作为默认值。

当遇到错误并将详细信息设置为 3 如下所示:

Verifying that GeoNode is running ...
Found 8 potential layers.
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp' (1/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_coastline.shp' (2/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_highway.shp' (3/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_location.shp' (4/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_natural.shp' (5/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_poi.shp' (6/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_water.shp' (7/8)
[failed] Layer for '/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/single_point.shp' (8/8)

Detailed report of failures:

/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp
================
Traceback (most recent call last):
  File "/Users/geosolutions/projects/geonode/geonode/layers/utils.py", line 682, in upload
    keywords=keywords,
  File "/Users/geosolutions/projects/geonode/geonode/layers/utils.py", line 602, in file_upload
    keywords=keywords, title=title)
  File "/Users/geosolutions/projects/geonode/geonode/layers/utils.py", line 305, in save
    store = cat.get_store(name)
  File "/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 176, in get_store
    for ws in self.get_workspaces():
  File "/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 489, in get_workspaces
    description = self.get_xml("%s/workspaces.xml" % self.service_url)
  File "/Users/geosolutions/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 136, in get_xml
    response, content = self.http.request(rest_url)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1445, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1197, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1133, in _conn_request
    conn.connect()
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 799, in connect
    raise socket.error, msg
error: [Errno 61] Connection refused

注解

输出的最后一部分将对所有层重复,上面只显示第一部分。

此错误表示geonode无法连接到geoserver以加载层。要解决此问题,应确保geoserver正在运行,然后重新运行该命令。

如果遇到此命令无法解决的错误,应将其显示在geonode用户邮件列表中。

现在,您应该具备了将层从服务器文件系统上的目录导入geonode项目所需的知识,并且可以使用它一次将许多层加载到geonode中。

注解

如果你不使用 -u 命令选项,导入图层的所有权将分配给系统中的主要超级用户。如果希望geonodes django管理接口归另一个用户所有,则可以在事后使用geonodes django管理接口对此进行修改。

管理指挥部 updatelayers

虽然可以直接从服务器服务器文件系统层导入GeoNoDE,但您可能有一个已经存在数据的GeoServer,或者您可能想从GeoServer中配置数据,而GeoServer不直接通过上传数据来支持。

geoserver支持多种数据格式和到数据库的连接,虽然其中许多格式不支持作为geonode上载格式,但如果可以在geoserver中配置它们,则可以按照下面描述的过程将它们添加到geonode中。

GeoServer支持3种类型的数据: RasterVectorDatabasesCascaded .

有关每种类型数据支持格式的列表,请参阅以下页面:

注解

其中一些栅格或矢量格式或数据库类型需要在geoserver中安装特定的插件才能使用。有关详细信息,请参阅geoserver文档。

postgis数据库中的数据

让我们看一个在geoserver中配置新postgis数据库,然后在geonode中配置这些层的示例。

首先访问服务器上的geoserver管理界面。这通常在端口8080上,可在http://localhost:8080/geoserver/web上获得/

  1. 您应该使用第一次配置geonode实例时设置的超级用户凭据登录。

    登录到geoserver管理界面后,应看到以下内容。

    ../../_images/geoserver_admin.png

    注解

    存储、层和工作区的数量可能不同,具体取决于您在geoserver中已配置的内容。

  2. 接下来您要选择左侧菜单中的“商店”选项,然后选择“添加新商店”选项。将显示以下屏幕。

    ../../_images/geoserver_new_store.png
  3. 在这种情况下,我们希望选择PASGIS存储类型来创建与现有数据库的连接。在下一个屏幕上,您将需要输入连接到postgis数据库的参数(根据需要更改您自己的数据库)。

    ../../_images/geoserver_postgis_params.png

    注解

    如果您不确定任何设置,请将其保留为默认设置。

  4. 下一个屏幕允许您配置数据库中的层。当然,这取决于数据库中的层。

    ../../_images/geoserver_publish_layers.png
  5. 选择其中一个图层的“发布”按钮,将显示下一个屏幕,您可以在其中输入此图层的元数据。因为我们将在geonode中管理此元数据,所以现在可以不使用这些元数据。

    ../../_images/geoserver_layer_params.png
  6. 那些 must be specified是声明的srs,必须在指定srs之后选择“compute from data”和“compute from native bounds”链接。

    ../../_images/geoserver_srs.png
    ../../_images/geoserver_srs_2.png
  7. 单击“保存”,此图层现在将配置为在地理服务器中使用。

    ../../_images/geoserver_layers.png
  8. 下一步是在geonode中配置这些层。这个 updatelayers 管理命令可用于此目的。就像 importlayers ,通过传递 --help 选项

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers --help
    

    注解

    如果启用 local_settings.py 命令将更改如下:

    DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py updatelayers --help
    

    这将产生如下输出

    usage: manage.py updatelayers [-h] [--version] [-v {0,1,2,3}]
                                [--settings SETTINGS] [--pythonpath PYTHONPATH]
                                [--traceback] [--no-color] [-i]
                                [--skip-unadvertised]
                                [--skip-geonode-registered] [--remove-deleted]
                                [-u USER] [-f FILTER] [-s STORE] [-w WORKSPACE]
                                [-p PERMISSIONS]
    
    Update the GeoNode application with data from GeoServer
    
    optional arguments:
    -h, --help            show this help message and exit
    --version             show program's version number and exit
    -v {0,1,2,3}, --verbosity {0,1,2,3}
                            Verbosity level; 0=minimal output, 1=normal output,
                            2=verbose output, 3=very verbose output
    --settings SETTINGS   The Python path to a settings module, e.g.
                            "myproject.settings.main". If this isn't provided, the
                            DJANGO_SETTINGS_MODULE environment variable will be
                            used.
    --pythonpath PYTHONPATH
                            A directory to add to the Python path, e.g.
                            "/home/djangoprojects/myproject".
    --traceback           Raise on CommandError exceptions
    --no-color            Don't colorize the command output.
    -i, --ignore-errors   Stop after any errors are encountered.
    --skip-unadvertised   Skip processing unadvertised layers from GeoSever.
    --skip-geonode-registered
                            Just processing GeoServer layers still not registered
                            in GeoNode.
    --remove-deleted      Remove GeoNode layers that have been deleted from
                            GeoSever.
    -u USER, --user USER  Name of the user account which should own the imported
                            layers
    -f FILTER, --filter FILTER
                            Only update data the layers that match the given
                            filter
    -s STORE, --store STORE
                            Only update data the layers for the given geoserver
                            store name
    -w WORKSPACE, --workspace WORKSPACE
                            Only update data on specified workspace
    -p PERMISSIONS, --permissions PERMISSIONS
                            Permissions to apply to each layer
    

警告

其中之一 --workspace--store 如果要摄取属于特定 Workspace 是的。作为一个实例,为了将存在于 geonode 工作区,则需要指定选项 -w geonode .

  1. 让我们吃下这层 geonode:_1_SARMIENTO_ENERO_2018geonode 工作区。

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers -w geonode -f _1_SARMIENTO_ENERO_2018
    
    Inspecting the available layers in GeoServer ...
    Found 1 layers, starting processing
    /usr/local/lib/python2.7/site-packages/owslib/iso.py:117: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties.  This is currently implemented in .identificationinfo.  Please see https://github.com/geopython/OWSLib/issues/38 for more information
    FutureWarning)
    /usr/local/lib/python2.7/site-packages/owslib/iso.py:495: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information
    FutureWarning)
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A new layer has been uploaded
    From: webmaster@localhost
    To: mapadeldelito@chubut.gov.ar
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:17 -0000
    Message-ID: <20191008122617.28801.94967@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The user <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i> uploaded the following layer:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A new layer has been uploaded
    From: webmaster@localhost
    To: giacomo8vinci@gmail.com
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:17 -0000
    Message-ID: <20191008122617.28801.53784@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The user <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i> uploaded the following layer:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A new layer has been uploaded
    From: webmaster@localhost
    To: fmgagliano@gmail.com
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:17 -0000
    Message-ID: <20191008122617.28801.26265@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The user <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i> uploaded the following layer:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Found geoserver resource for this layer: _1_SARMIENTO_ENERO_2018
    ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018]
    -- Resource Links[Prune old links]...
    -- Resource Links[Prune old links]...done!
    -- Resource Links[Compute parameters for the new links]...
    -- Resource Links[Create Raw Data download link]...
    -- Resource Links[Create Raw Data download link]...done!
    -- Resource Links[Set download links for WMS, WCS or WFS and KML]...
    -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done!
    -- Resource Links[Legend link]...
    -- Resource Links[Legend link]...done!
    -- Resource Links[Thumbnail link]...
    -- Resource Links[Thumbnail link]...done!
    -- Resource Links[OWS Links]...
    -- Resource Links[OWS Links]...done!
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A layer has been updated
    From: webmaster@localhost
    To: mapadeldelito@chubut.gov.ar
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:20 -0000
    Message-ID: <20191008122620.28801.81598@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The following layer was updated:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong>, owned by <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A layer has been updated
    From: webmaster@localhost
    To: giacomo8vinci@gmail.com
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:20 -0000
    Message-ID: <20191008122620.28801.93778@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The following layer was updated:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong>, owned by <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Content-Type: text/html; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: [master.demo.geonode.org] A layer has been updated
    From: webmaster@localhost
    To: fmgagliano@gmail.com
    Reply-To: webmaster@localhost
    Date: Tue, 08 Oct 2019 12:26:20 -0000
    Message-ID: <20191008122620.28801.58585@d3cf85425231>
    
    
    <body>
    You have received the following notice from master.demo.geonode.org:
    <p>
    
    The following layer was updated:<br/>
    <strong>_1_SARMIENTO_ENERO_2018</strong>, owned by <i><a href="http://master.demo.geonode.org/people/profile/admin">admin</a></i><br/>
    You can visit the layer's detail page here: http://master.demo.geonode.org/layers/geonode:_1_SARMIENTO_ENERO_2018
    
    </p>
    <p>
    To change how you receive notifications, please go to http://master.demo.geonode.org
    </p>
    </body>
    
    -------------------------------------------------------------------------------
    Found geoserver resource for this layer: _1_SARMIENTO_ENERO_2018
    /usr/local/lib/python2.7/site-packages/geoserver/style.py:80: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if not user_style:
    /usr/local/lib/python2.7/site-packages/geoserver/style.py:84: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if user_style:
    ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018]
    -- Resource Links[Prune old links]...
    -- Resource Links[Prune old links]...done!
    -- Resource Links[Compute parameters for the new links]...
    -- Resource Links[Create Raw Data download link]...
    -- Resource Links[Create Raw Data download link]...done!
    -- Resource Links[Set download links for WMS, WCS or WFS and KML]...
    -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done!
    -- Resource Links[Legend link]...
    -- Resource Links[Legend link]...done!
    -- Resource Links[Thumbnail link]...
    -- Resource Links[Thumbnail link]...done!
    -- Resource Links[OWS Links]...
    -- Resource Links[OWS Links]...done!
    [created] Layer _1_SARMIENTO_ENERO_2018 (1/1)
    
    
    Finished processing 1 layers in 5.0 seconds.
    
    1 Created layers
    0 Updated layers
    0 Failed layers
    5.000000 seconds per layer
    

注解

如果您没有指定 -f 选项,在GeoNoDE中已经存在的层将被更新,GeoServer和GeoNode之间的配置同步。

警告

更新时 from GeoServer上的配置将被更改!

使用 GDALOGR 转换数据以在geonode中使用

geonode支持在 ESRI shapefilesGeoTIFFCSVGeoJSONASCII-GRIDKML / KMZ 格式(仅当使用 geonode.importer 后端)。

  • 如果数据采用其他格式,则需要将其转换为这些格式之一,以便在geonode中使用。

  • 如果你 Raster 数据未正确处理,它可能在geoserver和geonode中几乎不可用。你需要用 GDAL .

您需要确保在系统上安装了gdal库。在ubuntu上你可以用以下命令安装这个包:

sudo apt-get install gdal-bin

OGR(矢量数据)

ogr用于处理矢量数据。在本例中,我们将使用mapinfo.tab文件,并使用ogr2ogr命令将它们转换为shapefile。我们将使用下面链接的网站中的示例mapinfo文件。

http://services.land.vic.gov.au/landchannel/content/help?name=sampledata

您可以通过发出以下命令下载admin;(postcode)层:

$ wget http://services.land.vic.gov.au/sampledata/shape/admin_postcode_vm.zip

您需要通过发出以下命令来解压缩此数据集:

$ unzip admin_postcode_vm.zip

这将在执行上述命令的目录中留下以下文件:

|-- ANZVI0803003025.htm
|-- DSE_Data_Access_Licence.pdf
|-- VMADMIN.POSTCODE_POLYGON.xml
|-- admin_postcode_vm.zip
--- vicgrid94
    --- mif
        --- lga_polygon
            --- macedon\ ranges
                |-- EXTRACT_POLYGON.mid
                |-- EXTRACT_POLYGON.mif
                --- VMADMIN
                    |-- POSTCODE_POLYGON.mid
                    --- POSTCODE_POLYGON.mif

首先,让我们使用以下命令检查此文件集:

$ ogrinfo -so vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON

输出如下所示:

Had to open data source read-only.
INFO: Open of `vicgrid94/mif/lga_polygon/macedon ranges/VMADMIN/POSTCODE_POLYGON.mid'
    using driver `MapInfo File' successful.

Layer name: POSTCODE_POLYGON
Geometry: 3D Unknown (any)
Feature Count: 26
Extent: (2413931.249367, 2400162.366186) - (2508952.174431, 2512183.046927)
Layer SRS WKT:
PROJCS["unnamed",
    GEOGCS["unnamed",
        DATUM["GDA94",
            SPHEROID["GRS 80",6378137,298.257222101],
            TOWGS84[0,0,0,-0,-0,-0,0]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433]],
    PROJECTION["Lambert_Conformal_Conic_2SP"],
    PARAMETER["standard_parallel_1",-36],
    PARAMETER["standard_parallel_2",-38],
    PARAMETER["latitude_of_origin",-37],
    PARAMETER["central_meridian",145],
    PARAMETER["false_easting",2500000],
    PARAMETER["false_northing",2500000],
    UNIT["Meter",1]]
PFI: String (10.0)
POSTCODE: String (4.0)
FEATURE_TYPE: String (6.0)
FEATURE_QUALITY_ID: String (20.0)
PFI_CREATED: Date (10.0)
UFI: Real (12.0)
UFI_CREATED: Date (10.0)
UFI_OLD: Real (12.0)

这将为您提供有关此图层的要素数量、范围、投影和属性的信息。

接下来,让我们执行以下命令,将此层转换为shapefile:

$ ogr2ogr -t_srs EPSG:4326 postcode_polygon.shp vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON

请注意,我们还使用-t_srs ogr2ogr选项将该层重新投影到wgs84空间参考系统。

此命令的输出如下所示:

Warning 6: Normalized/laundered field name: 'FEATURE_TYPE' to 'FEATURE_TY'
Warning 6: Normalized/laundered field name: 'FEATURE_QUALITY_ID' to 'FEATURE_QU'
Warning 6: Normalized/laundered field name: 'PFI_CREATED' to 'PFI_CREATE'
Warning 6: Normalized/laundered field name: 'UFI_CREATED' to 'UFI_CREATE'

此输出指示某些字段名已被截断,以符合shapefile中的属性只有10个字符长的约束。

现在您将拥有一组文件,这些文件构成postcode_polygon.shp shapefile集。我们可以通过发出以下命令来检查它们:

$ ogrinfo -so postcode_polygon.shp postcode_polygon

当我们检查从以下位置转换的mapinfo文件时,输出将类似于我们在上面看到的输出:

INFO: Open of `postcode_polygon.shp'
      using driver `ESRI Shapefile' successful.

Layer name: postcode_polygon
Geometry: Polygon
Feature Count: 26
Extent: (144.030296, -37.898156) - (145.101137, -36.888878)
Layer SRS WKT:
GEOGCS["GCS_WGS_1984",
    DATUM["WGS_1984",
        SPHEROID["WGS_84",6378137,298.257223563]],
    PRIMEM["Greenwich",0],
    UNIT["Degree",0.017453292519943295]]
PFI: String (10.0)
POSTCODE: String (4.0)
FEATURE_TY: String (6.0)
FEATURE_QU: String (20.0)
PFI_CREATE: Date (10.0)
UFI: Real (12.0)
UFI_CREATE: Date (10.0)
UFI_OLD: Real (12.0)

这些文件现在可以通过普通上传程序加载到geonode实例中。

访问geonode中的upload页面,拖放使用gdal ogr2ogr命令(postcode_polygon.dbf,postcode_polygon.prj,postcode_polygon.shp,postcode_polygon.shx)生成的shapefile文件。根据需要授予权限,然后单击“上载文件”按钮。

../../_images/upload_shapefile.png

一旦导入过程完成,您就可以直接进入图层信息页面(“图层信息”按钮),或编辑该图层的元数据(“编辑元数据”按钮),或管理该图层的样式(“管理样式”)。

../../_images/layer_info_vector.png

GDAL(栅格数据)

让我们看几个关于如何将栅格数据转换为不同格式和/或处理它以获得最佳性能的示例。

参考文献:

  1. https://geoserver.geo-solutions.it/edu/en/raster_data/processing.html

  2. https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/

栅格数据转换:将arc/info二进制和ascii网格数据转换为geotiff格式。

假设我们有一个压缩为存档的ascii网格文件示例。

# Un-tar the files
tar -xvf sample_asc.tar

您的文件系统中将保留以下文件:

|-- batemans_ele
|   |-- dblbnd.adf
|   |-- hdr.adf
|   |-- metadata.xml
|   |-- prj.adf
|   |-- sta.adf
|   |-- w001001.adf
|   |-- w001001x.adf
|-- batemans_elevation.asc

文件 batemans_elevation.asc 是一个arc/info ascii网格文件,batemans_ele目录中的文件是一个arc/info二进制网格文件。

你可以使用 gdalinfo 命令通过执行以下命令检查这两个文件:

gdalinfo batemans_elevation.asc

输出应如下所示:

Driver: AAIGrid/Arc/Info ASCII Grid
Files: batemans_elevation.asc
Size is 155, 142
Coordinate System is `'
Origin = (239681.000000000000000,6050551.000000000000000)
Pixel Size = (100.000000000000000,-100.000000000000000)
Corner Coordinates:
Upper Left  (  239681.000, 6050551.000)
Lower Left  (  239681.000, 6036351.000)
Upper Right (  255181.000, 6050551.000)
Lower Right (  255181.000, 6036351.000)
Center      (  247431.000, 6043451.000)
Band 1 Block=155x1 Type=Float32, ColorInterp=Undefined
    NoData Value=-9999

然后,您可以通过执行以下命令检查batemans_ele文件:

gdalinfo batemans_ele

这应该是相应的输出:

Driver: AIG/Arc/Info Binary Grid
Files: batemans_ele
    batemans_ele/dblbnd.adf
    batemans_ele/hdr.adf
    batemans_ele/metadata.xml
    batemans_ele/prj.adf
    batemans_ele/sta.adf
    batemans_ele/w001001.adf
    batemans_ele/w001001x.adf
Size is 155, 142
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["GDA94",
        DATUM["Geocentric_Datum_of_Australia_1994",
            SPHEROID["GRS 1980",6378137,298.257222101,
                AUTHORITY["EPSG","7019"]],
            TOWGS84[0,0,0,0,0,0,0],
            AUTHORITY["EPSG","6283"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4283"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",153],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",10000000],
    UNIT["METERS",1]]
Origin = (239681.000000000000000,6050551.000000000000000)
Pixel Size = (100.000000000000000,-100.000000000000000)
Corner Coordinates:
Upper Left  (  239681.000, 6050551.000) (150d 7'28.35"E, 35d39'16.56"S)
Lower Left  (  239681.000, 6036351.000) (150d 7'11.78"E, 35d46'56.89"S)
Upper Right (  255181.000, 6050551.000) (150d17'44.07"E, 35d39'30.83"S)
Lower Right (  255181.000, 6036351.000) (150d17'28.49"E, 35d47'11.23"S)
Center      (  247431.000, 6043451.000) (150d12'28.17"E, 35d43'13.99"S)
Band 1 Block=256x4 Type=Float32, ColorInterp=Undefined
    Min=-62.102 Max=142.917
NoData Value=-3.4028234663852886e+38

你会注意到 batemans_elevation.asc 文件可以 not 包含投影信息,而 batemans_ele 文件可以。因此,让我们用 batemans_ele 用于此练习的文件,并将它们转换为geotiff以在geonode中使用。在此过程中,我们还会将此文件重新投影到wgs84中。这可以通过以下命令完成。

gdalwarp -t_srs EPSG:4326 batemans_ele batemans_ele.tif

输出将显示转换的进度,当转换完成时,将留给您一个 batemans_ele.tif 可以上载到geonode的文件。

可以使用gdalinfo命令检查此文件:

gdalinfo batemans_ele.tif

将产生以下输出:

Driver: GTiff/GeoTIFF
Files: batemans_ele.tif
Size is 174, 130
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (150.119938943722502,-35.654598806259330)
Pixel Size = (0.001011114155919,-0.001011114155919)
Metadata:
    AREA_OR_POINT=Area
Image Structure Metadata:
    INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( 150.1199389, -35.6545988) (150d 7'11.78"E, 35d39'16.56"S)
Lower Left  ( 150.1199389, -35.7860436) (150d 7'11.78"E, 35d47' 9.76"S)
Upper Right ( 150.2958728, -35.6545988) (150d17'45.14"E, 35d39'16.56"S)
Lower Right ( 150.2958728, -35.7860436) (150d17'45.14"E, 35d47' 9.76"S)
Center      ( 150.2079059, -35.7203212) (150d12'28.46"E, 35d43'13.16"S)
Band 1 Block=174x11 Type=Float32, ColorInterp=Gray

栅格数据优化:优化和服务大栅格数据

(参考号:https://geoserver.geo-solutions.it/edu/en/raster-data/advanced-gdal/example5.html

在处理大型栅格数据集时,使用平铺可能非常有用。

平铺允许将大型栅格数据集分解为可管理的部分,这是定义和实现更高级别栅格I/O接口的基础。

在本例中,我们将使用 chiangMai_ortho_optimized 公共栅格层,目前在泰国 CHIANG MAI Urban Flooding GeoNode platform .

此数据集包含存储为rgba geotiff的正射图像,其中4个波段、3个波段用于rgb,1个用于透明度(alpha通道)。

调用gdalinfo命令查看详细信息:

gdalinfo chiangMai_ortho.tif

它将产生以下结果:

Driver: GTiff/GeoTIFF
Files: chiangMai_ortho.tif
Size is 63203, 66211
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.028850000000000,-0.028850000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N)
Lower Right (  488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=63203x1 Type=Byte, ColorInterp=Red
NoData Value=-10000
Mask Flags: PER_DATASET ALPHA
Band 2 Block=63203x1 Type=Byte, ColorInterp=Green
NoData Value=-10000
Mask Flags: PER_DATASET ALPHA
Band 3 Block=63203x1 Type=Byte, ColorInterp=Blue
NoData Value=-10000
Mask Flags: PER_DATASET ALPHA
Band 4 Block=63203x1 Type=Byte, ColorInterp=Alpha
NoData Value=-10000

如您所见,这个geotiff没有被平铺。但是,对于访问子集,平铺可以产生影响。使用平铺,数据以块(平铺)的形式存储和压缩,而不是逐行(剥离)。

在上面的命令输出中,可以看到每个频带都有具有相同图像宽度(63203)和单位长度的块。下图中的网格显示一个具有相同大小的平铺(左)和相同数量的条带(右)的图像。要从红色子集读取数据,必须解压缩相交区域。

../../_images/tiled_vs_stripped.png

在平铺图像中,我们只需要解压缩16个平铺,而在右边的平铺图像中,我们需要解压缩更多的平铺。

无人机图像数据通常具有剥离结构,因此,在大多数情况下,需要对其进行优化以提高性能。

让我们看看用于优化geotiff的gdal_translate命令:

gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR
            --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4
            chiangMai_ortho.tif
            chiangMai_ortho_optimized.tif

注解

有关命令参数的详细信息,请参阅https://geoserver.geo-solutions.it/edu/en/raster-data/advanced-gdal/example5.html

进程结束后,对生成的tif文件调用gdalinfo命令:

gdalinfo chiangMai_ortho_optimized.tif

结果如下:

Driver: GTiff/GeoTIFF
Files: chiangMai_ortho_optimized.tif
Size is 63203, 66211
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.028850000000000,-0.028850000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=YCbCr JPEG
INTERLEAVE=PIXEL
SOURCE_COLOR_SPACE=YCbCr
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N)
Lower Right (  488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
NoData Value=-10000
Mask Flags: PER_DATASET
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
NoData Value=-10000
Mask Flags: PER_DATASET
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue
NoData Value=-10000
Mask Flags: PER_DATASET

我们的geotiff现在是用256x256瓷砖平铺的,有3个波段和1位nodata掩模。

我们还可以使用gdaladdo命令将内部概述添加到文件中:

gdaladdo -r average chiangMai_ortho_optimized.tif 2 4 8 16 32 64 128 256 512

概述是原始数据的重复版本,但是重新采样到较低的分辨率时,也可以使用各种算法对其进行压缩,这与原始数据集的压缩方式非常相似。

默认情况下,overview采用与输入数据集相同的压缩类型和透明度掩码(通过gdal_translate命令应用),因此要指定的参数为:

  • -r average :计算所有非nodata贡献像素的平均值

  • 2 4 8 16 32 64 128 256 512 :要生成的整体概述级别列表(不再需要从gdal版本2.3级别生成概述)

再次调用gdalinfo命令:

gdalinfo chiangMai_ortho_optimized.tif

结果是:

Driver: GTiff/GeoTIFF
Files: chiangMai_ortho_optimized.tif
Size is 63203, 66211
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.028850000000000,-0.028850000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=YCbCr JPEG
INTERLEAVE=PIXEL
SOURCE_COLOR_SPACE=YCbCr
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N)
Lower Right (  488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
NoData Value=-10000
Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130
Mask Flags: PER_DATASET
Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
NoData Value=-10000
Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130
Mask Flags: PER_DATASET
Overviews of mask band: 31602x3Results in:3106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue
NoData Value=-10000
Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130
Mask Flags: PER_DATASET
Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130

请注意,已应用内部概述的透明度掩码(它们的压缩不会显示在文件元数据中)。

无人机通常还提供两种其他类型的数据: DTM (Digital Terrain Model)DSM (Digital Surface Model) .

这些数据需要优化不同的过程。让我们看看一些例子,以便更好地理解如何使用gdal来完成该任务。

CHIANG MAI Urban Flooding GeoNode platform 当前可用的平台 chiangMai_dtm_optimized 层,让我们下载它的原始数据集。

此数据集应包含DTM文件 chiangMai_dtm.tif .

对其调用gdalinfo命令:

gdalinfo chiangMai_dtm.tif

将显示以下信息:

Driver: GTiff/GeoTIFF
Files: chiangMai_dtm.tif
Size is 12638, 13240
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.144270000000000,-0.144270000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N)
Lower Right (  488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=12638x1 Type=Float32, ColorInterp=Gray
NoData Value=-10000

读取此图像可能非常慢,因为它尚未平铺。因此,如上所述,需要将其数据存储并压缩到分片中以提高性能。

以下gdal_translate命令应适用于此目的:

gdal_translate -co TILED=YES -co COMPRESS=DEFLATE chiangMai_dtm.tif chiangMai_dtm_optimized.tif

当要压缩的数据包含图像时。航空照片、真彩色卫星图像或彩色地图)可以使用有损算法,如JPEG。我们现在正在压缩精度很重要的数据,频带数据类型是float32,高程值不应该改变,所以像jpeg这样的有损算法是不合适的。jpeg通常只能用于字节数据(每个通道8位),因此我们通过compress=deflate创建选项选择了无损的deflate压缩。

再次调用gdalinfo命令:

gdalinfo chiangMai_dtm_optimized.tif

我们可以观察到以下结果:

Driver: GTiff/GeoTIFF
Files: chiangMai_dtm_optimized.tif
Size is 12638, 13240
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.144270000000000,-0.144270000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=DEFLATE
INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N)
Lower Right (  488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
NoData Value=-10000

我们还需要通过gdaladdo命令创建概述:

gdaladdo -r nearest chiangMai_dtm_optimized.tif 2 4 8 16 32 64

与前面的示例不同,将使用 最近重采样算法 是的。这是由于我们所代表的数据的性质:我们不应考虑两个高程值之间的平均值,而只应考虑越接近的高程值,这对于原始数据的保存更为可靠。

再次调用gdalinfo命令:

gdalinfo chiangMai_dtm_optimized.tif

我们可以看到以下信息:

Driver: GTiff/GeoTIFF
Files: chiangMai_dtm_optimized.tif
Size is 12638, 13240
Coordinate System is:
PROJCS["WGS 84 / UTM zone 47N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",99],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32647"]]
Origin = (487068.774750000040513,2057413.889810000080615)
Pixel Size = (0.144270000000000,-0.144270000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=DEFLATE
INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N)
Lower Left  (  487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N)
Upper Right (  488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N)
Lower Right (  488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N)
Center      (  487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
NoData Value=-10000
Overviews: 6319x6620, 3160x3310, 1580x1655, 790x828, 395x414, 198x207

已创建概述。默认情况下,它们继承原始数据集的相同压缩类型(在gdalinfo输出中没有证据)。

以编程方式处理栅格数据集

在本节中,我们将提供一组 shell 脚本对于以编程方式批量处理大量栅格数据集可能非常有用。

  1. process_gray.sh

    for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh
    chmod +x gdal_translate.sh
    ./gdal_translate.sh
    
    for filename in *.optimized.tif*; do echo gdaladdo -r nearest $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh
    for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh
    chmod +x *.sh
    ./gdaladdo.sh
    ./rename.sh
    
  2. process_rgb.sh

    for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -b 1 -b 2 -b 3 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh
    chmod +x gdal_translate.sh
    ./gdal_translate.sh
    
    for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh
    for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh
    chmod +x *.sh
    ./gdaladdo.sh
    ./rename.sh
    
  3. process_rgb_alpha.sh

    for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh
    chmod +x gdal_translate.sh
    ./gdal_translate.sh
    
    for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh
    for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh
    chmod +x *.sh
    ./gdaladdo.sh
    ./rename.sh
    
  4. process_rgb_palette.sh

    for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh
    chmod +x gdal_translate.sh
    ./gdal_translate.sh
    
    for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh
    for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh
    chmod +x *.sh
    ./gdaladdo.sh
    ./rename.sh
    

创建用户和超级用户

第一步是创建一个用户。有三个选项可供选择,具体取决于要创建的用户类型,您可以选择不同的选项。我们将从创建 超级用户 ,因为这个用户是最重要的。超级用户拥有所有权限,而无需显式分配这些权限。

创建超级用户(在Linux中)的最简单方法是打开终端并键入:

$ DJANGO_SETTINGS_MODULE=geonode.settings python manage.py createsuperuser

注解

如果启用 local_settings.py 命令将更改如下:

$ DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py createsuperuser

将询问您一个用户名(在本教程中,我们将调用您现在创建的超级用户 your_superuser ),电子邮件地址和密码。

现在您已经创建了一个超级用户,您应该熟悉 Django管理界面 是的。作为超级用户,您可以访问此界面,在这里您可以管理用户、层、权限等。若要了解有关此接口的详细信息,请检查此链接。现在只要按照步骤走就足够了。参加 Django管理界面 ,转到geonode网站并 登录 具有 your_superuser 是的。登录后,用户的名称将显示在右上角。单击它,将显示以下菜单:

../../_images/menu_admin.png

点击 行政 使界面出现。

../../_images/admin_interface.png

Auth > 用户 你会看到目前所有的用户。如果是你的话 your_superuser 是的。单击它,您将看到 个人信息 ,打开一个 权限 还有一个 重要日期 是的。就目前而言 权限 是最重要的。

../../_images/permissions_django_admin.png

如您所见,有三个框可以选中和取消选中。因为您已经创建了一个超级用户,所以所有这三个框都将作为默认值选中。如果只有盒子 积极的 将被选中,该用户将不是超级用户,并且将无法访问 Django管理界面 (仅适用于具有 工作人员 状态)。因此,请记住以下两点:

  • 超级用户可以访问 Django管理界面 他对上传到geonode的数据拥有所有权限。

  • 普通用户(从geonode接口创建)只有 积极的 默认情况下为权限。用户将无法访问 Django管理界面 必须为他添加某些权限。

到目前为止,我们只创建了超级用户。那么如何创建一个普通用户呢?你有两个选择:

  1. Django管理界面

    首先,我们将通过 Django管理界面 因为我们还开着呢。所以回到 Auth > 用户 你应该在右边找到一个按钮 添加用户 .

    ../../_images/add_user.png

    点击它,就会出现一个要填写的表格。命名新用户test_user,选择密码并单击 save 在网站的右下方。

    ../../_images/add_test_user.png

    现在您应该被引导到可以更改用户权限的站点 test_user 是的。仅作为默认值 积极的 已检查。如果您希望此用户也能够参加此管理界面,也可以检查 工作人员状况 是的。但现在我们还是保持原样!

    要测试是否成功创建了新用户,请返回geonode网页并尝试登录。

  2. 地理节点网站

    要创建普通用户,也可以使用geonode网站。如果使用发行版安装geonode,则应

    见a 寄存器 按钮在顶部,旁边 登录 按钮(您可能必须先注销)。

    ../../_images/register.png

    按一下按钮,就会出现一个表格供您填写。此用户将被命名为 geonode_user

    ../../_images/sign_up_test_user.png

    通过打击 注册 用户将被注册,默认情况下只有 积极的 .

批同步权限

GeoNode提供了一个非常有用的管理命令 set_layers_permisions 允许管理员轻松添加/删除对一个或多个层上的组和用户的权限。

这个 set_layers_permisions 命令参数为:

  • 权限 设置/取消设置-->读(r)、写(w)、下载(d)、所有者(o)

    READ_PERMISSIONS = [
        'view_resourcebase'
    ]
    WRITE_PERMISSIONS = [
        'change_layer_data',
        'change_layer_style',
        'change_resourcebase_metadata'
    ]
    DOWNLOAD_PERMISSIONS = [
        'download_resourcebase'
    ]
    OWNER_PERMISSIONS = [
        'change_resourcebase',
        'delete_resourcebase',
        'change_resourcebase_permissions',
        'publish_resourcebase'
    ]
    
  • 资源 (layers)哪些权限将被分配给-->键入层标题(对带空格的标题使用引号),可以用空格分隔符键入多选,如果没有提供标题,将考虑所有层

  • 用户 将为可以键入多个选项的用户分配空白分隔符

  • 将为可以键入多个选项的用户分配空白分隔符

  • 删除 标志(可选),表示将取消设置权限

使用示例:

  1. 赋值 层上的权限 layer_XY层 对用户 user_Auser_B 对整个团队来说 group_C .

    python manage.py set_layers-permissions -p write -u user_A user_B -g group_C -r layer_X 'layer Y'
    
  2. 赋值 主人 对组的所有层的权限 group_C .

    python manage.py set_layers-permissions -p owner -g group_C
    
  3. 未设置 下载 层上的权限 layer_X 对于用户 user_A .

    python manage.py set_layers-permissions -p download -u user_A -r layer_X -d
    

也可以从 Admin Dashboard >> Layers .

../../_images/layer_batch_perms_admin.png

一个名为 Set layers permissions 可从列表中获取,将管理员重定向到表单以设置/取消设置所选层的读、写、下载和所有权权限。

../../_images/layer_batch_perms_form.png

删除某些GeoNode资源

这个 delete_resources Management Command 允许移除以序列化django Q()表达式形式指定的满足特定条件的资源。

首先让我们看看 --help 选择权 delete_resources 管理命令,以检查所有命令选项和功能。

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources --help

注解

如果启用 local_settings.py 命令将更改如下:

DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py delete_resources --help

这将产生以下输出:

usage: manage.py delete_resources [-h] [-c CONFIG_PATH]
                                  [-l LAYER_FILTERS [LAYER_FILTERS ...]]
                                  [-m MAP_FILTERS [MAP_FILTERS ...]]
                                  [-d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...]]
                                  [--version] [-v {0,1,2,3}]
                                  [--settings SETTINGS]
                                  [--pythonpath PYTHONPATH] [--traceback]
                                  [--no-color] [--force-color]

Delete resources meeting a certain condition

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_PATH, --config CONFIG_PATH
                        Configuration file path. Default is:
                        delete_resources.json
  -l LAYER_FILTERS [LAYER_FILTERS ...], --layer_filters LAYER_FILTERS [LAYER_FILTERS ...]
  -m MAP_FILTERS [MAP_FILTERS ...], --map_filters MAP_FILTERS [MAP_FILTERS ...]
  -d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...], --document_filters DOCUMENT_FILTERS [DOCUMENT_FILTERS ...]
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --force-color         Force colorization of the command output.

有两种方法可以声明Q()表达式来过滤应删除哪些资源:

  1. 使用JSON配置文件:传递 -c 参数,指定JSON配置文件的路径。

  • 例1 :配置文件的相对路径(到 manage.py

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c geonode/base/management/commands/delete_resources.json
    
  • 例2 :配置文件的绝对路径

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c /home/User/Geonode/configs/delete_resources.json
    
  1. 使用CLI:传递 -l -d -m 列出每个资源(层、文档、映射)的参数

  • 例3 :删除不带配置文件的资源

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -d '*' -m "Q(pk__in=[1, 2])"
    

配置文件

JSON配置文件应该包含一个 filters 对象,它包括 layermapdocument 列表。每个列表指定应用于相应查询集的筛选条件,定义将删除哪些项。过滤器被计算并直接插入到Django.filter()方法中,这意味着作为独立列表项出现的过滤器被视为and条件。创建或查询 | 应使用运算符。更多信息请查看Django [文档] (https://docs.djangoproject.com/en/3.0/topics/db/queries/#复杂-查找-with-q-objects)。唯一的例外是传递带有 '*' 这将导致删除资源的所有queryset。

  • 例4 :配置文件的示例内容,它将删除ID为1、2和3的层,这些层属于 admin 用户,以及所有定义的映射。

    {
      "filters": {
      "layer": [
          "Q(pk__in=[1, 2, 3]) | Q(title__icontains='italy')",
          "Q(user__name=admin)"
        ],
      "map": ["*"],
      "document": []
      }
    }
    

CLI

可以使用指定CLI配置 -l -d -m 列表参数,实际上是配置JSON文件的转换。 -l -d -m 参数的计算方式与过滤器.layer, 过滤器.map以及过滤器.document根据实例4。以下示例的结果将与示例4等效:

  • 例5 :示例CLI配置,它将删除ID为1、2和3的层以及所有映射。

    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2, 3]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -m '*'