Sign In Help
Schneider Electric
HelpLogin / Register

Invite a Co-worker

Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
You have entered an invalid email address. Please re-enter the email address.
This co-worker has already been invited to the Exchange portal. Please invite another co-worker.
Please enter email address
Send Invite Cancel

Invitation Sent

Your invitation was sent.Thanks for sharing Exchange with your co-worker.
Send New Invite Close
  • Home
  • Communities
  • :
  • SCADA & Telemetry Solutions
  • :
  • Geo SCADA Expert Forum
  • :
  • Maximum columns of a data table
Community Menu
  • Forums
    • By Topic
        • EcoStruxure Building
          • Field Devices Forum
        • EcoStruxure Power & Grid
          • Metering & Power Quality
        • EcoStruxure IT
          • EcoStruxure IT forum
        • Industrial Automation
          • Industry Automation and Control Forum
          • Alliance System Integrators Forum
          • Machine Solutions in the Digital Transformation
          • EcoStruxure Automation Expert / IEC 61499 Forum
          • Industrial Edge Computing Forum
          • Level and Pressure Instrumentation Forum
          • Modicon User Group
          • PLC Club Indonesia
          • SEE Automation Club Forum
          • Fabrika ve Makina Otomasyonu Çözümleri
          • Форум по промышленной автоматизации СНГ
          • Motorabgangstechnik & Antriebstechnik Forum
        • SCADA & Telemetry Solutions
          • Geo SCADA Expert Forum
          • SCADA and Telemetry Devices Forum
        • Power Distribution IEC
          • Power Distribution and Digital
          • Power Standards & Regulations
          • Solutions for Motor Management
          • Paneelbouw & Energie Distributie
          • Eldistribution & Fastighetsautomation
          • Elektrik Tasarım Dağıtım ve Uygulama Çözümleri
        • Power Distribution NEMA
          • US Utility Forum
        • Power Distribution Softwares
          • EcoStruxure Power Design Forum
          • LayoutFAST User Group Forum
        • Wireless Information Network Solutions
          • Instrument Area Network
          • Remote Monitoring
          • Tank Level Monitoring
          • Remote Data Collection
        • Solutions for your Business
          • Solutions for Food & Beverage Forum
          • Solutions for Healthcare Forum
    • By Segment
        • Food & Beverage
          • Solutions for Food & Beverage Forum
        • Healthcare
          • Solutions for Healthcare Forum
      • EcoStruxure Building
        • Field Devices Forum
      • EcoStruxure Power & Grid
        • Metering & Power Quality
      • EcoStruxure IT
        • EcoStruxure IT forum
      • Industrial Automation
        • Industry Automation and Control Forum
        • Alliance System Integrators Forum
        • Machine Solutions in the Digital Transformation
        • EcoStruxure Automation Expert / IEC 61499 Forum
        • Industrial Edge Computing Forum
        • Level and Pressure Instrumentation Forum
        • Modicon User Group
        • PLC Club Indonesia
        • SEE Automation Club Forum
        • Fabrika ve Makina Otomasyonu Çözümleri
        • Форум по промышленной автоматизации СНГ
        • Motorabgangstechnik & Antriebstechnik Forum
      • SCADA & Telemetry Solutions
        • Geo SCADA Expert Forum
        • SCADA and Telemetry Devices Forum
      • Power Distribution IEC
        • Power Distribution and Digital
        • Power Standards & Regulations
        • Solutions for Motor Management
        • Paneelbouw & Energie Distributie
        • Eldistribution & Fastighetsautomation
        • Elektrik Tasarım Dağıtım ve Uygulama Çözümleri
      • Power Distribution NEMA
        • US Utility Forum
      • Power Distribution Softwares
        • EcoStruxure Power Design Forum
        • LayoutFAST User Group Forum
      • Wireless Information Network Solutions
        • Instrument Area Network
        • Remote Monitoring
        • Tank Level Monitoring
        • Remote Data Collection
      • Solutions for your Business
        • Solutions for Food & Beverage Forum
        • Solutions for Healthcare Forum
      • Food & Beverage
        • Solutions for Food & Beverage Forum
      • Healthcare
        • Solutions for Healthcare Forum
  • Blogs
    • By Topic
        • EcoStruxure Power & Grid
          • Backstage Access Resources
        • Industrial Automation
          • Industry 4.0 Blog
          • Industrie du Futur France
        • SCADA & Telemetry Solutions
          • SCADA and Telemetry Blog
        • Power Distribution IEC
          • Power Events & Webinars
          • Power Foundations Blog
          • UK&I Residential and Commercial Electrical Distribution
        • Power Distribution NEMA
          • NEMA Power Foundations Blog
        • Power Distribution Softwares
          • EcoStruxure Power Design Blog
        • Solutions for your Business
          • Solutions for Food & Beverage Blog
          • Solutions for Healthcare Blog
          • Solutions for Retail Blog
          • Solutions for eMobility
          • Digital Hospital Solutions – UK&I
        • Knowledge Center
          • Industrial Automation How-to videos
          • Digital E-books
          • Success Stories Corner
        • Community experts & publishers
          • Publishers Community
    • By Segment
        • Food & Beverage
          • Solutions for Food & Beverage Blog
        • Healthcare
          • Solutions for Healthcare Blog
        • Retail
          • Solutions for Retail Blog
        • Retail
          • Solutions for eMobility
        • Retail
          • Digital Hospital Solutions – UK&I
      • EcoStruxure Power & Grid
        • Backstage Access Resources
      • Industrial Automation
        • Industry 4.0 Blog
        • Industrie du Futur France
      • SCADA & Telemetry Solutions
        • SCADA and Telemetry Blog
      • Power Distribution IEC
        • Power Events & Webinars
        • Power Foundations Blog
        • UK&I Residential and Commercial Electrical Distribution
      • Power Distribution NEMA
        • NEMA Power Foundations Blog
      • Power Distribution Softwares
        • EcoStruxure Power Design Blog
      • Solutions for your Business
        • Solutions for Food & Beverage Blog
        • Solutions for Healthcare Blog
        • Solutions for Retail Blog
        • Solutions for eMobility
        • Digital Hospital Solutions – UK&I
      • Knowledge Center
        • Industrial Automation How-to videos
        • Digital E-books
        • Success Stories Corner
      • Community experts & publishers
        • Publishers Community
      • Food & Beverage
        • Solutions for Food & Beverage Blog
      • Healthcare
        • Solutions for Healthcare Blog
      • Retail
        • Solutions for Retail Blog
      • Retail
        • Solutions for eMobility
      • Retail
        • Digital Hospital Solutions – UK&I
  • Ideas
        • Industrial Automation
          • Modicon Ideas & new features
        • SCADA & Telemetry Solutions
          • Geo SCADA Expert Ideas
          • SCADA and Telemetry Devices Ideas
  • Knowledge Center
    • Building Automation Knowledge Base
    • Industrial Automation Knowledge Base
    • Industrial Automation How-to videos
    • SCADA & Telemetry Solutions Knowledge Base
    • Digital E-books
    • Success Stories Corner
    • Power Talks
  • Events & Webinars
    • Innovation Talks
    • Innovation Summit
    • Let's Exchange Series
    • Technology Partners
  • Support
    • Ask Exchange
    • Leaderboard
    • Our Community Guidelines
    • Community User Guide
    • How-To & Best Practices
    • More
Join Now
How can we help?
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
52705members
Join Now
247942posts
Join Now

Maximum columns of a data table

Options
  • Subscribe to RSS Feed
  • Mark Topic as New
  • Mark Topic as Read
  • Float this Topic for Current User
  • Bookmark
  • Subscribe
  • Mute
  • Printer Friendly Page
Solved Go to Solution
Back to Geo SCADA Expert Forum
Solved
DavidSkilbeck
Lt. Commander DavidSkilbeck
Lt. Commander
‎2020-09-18 10:58 AM
0 Likes
7
802
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-18 10:58 AM

Maximum columns of a data table

Hi,

what is the maximum number of columns in a data table and where can you find this information?

Thanks,

Labels
  • SCADA
Tags (1)
  • Tags:
  • english
Share
Reply

Accepted Solutions
BevanWeiss
Sisko BevanWeiss
Sisko
‎2020-09-18 09:26 PM
0 Likes
6
796
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-18 09:26 PM

Re: Maximum columns of a data table

Do you mean columns, or do you mean rows?

 

If you are thinking that you will run into an issue with too many columns, then I think your design is flawed.  You should not have this many columns.  I suspect that you will run into performance issues long before you start to hit the theoretical limits of the number of columns.

 

SQL itself supports 255 columns, so this will be the absolute upper limit.

ODBC / OLEDB have similar limits.

 

In regards to the number of rows, I believe the RowId value is a 32-bit Integer, so that puts a cap at something a little over 2 million.  But the Server Configuration query limits will also put a cap on it.

 

If you're getting towards the limits of the row count, then you likely also have a flawed design.  Geo SCADA's Data Table functionality isn't really intended to be a replacement for a stand alone database.  You will almost certainly suffer from performance limitations if trying to query for so many records from Geo SCADA.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..

See Answer In Context

Tags (1)
  • Tags:
  • english
Share
Reply
  • All forum topics
  • Previous Topic
  • Next Topic
7 Replies 7
BevanWeiss
Sisko BevanWeiss
Sisko
‎2020-09-18 09:26 PM
0 Likes
6
797
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-18 09:26 PM

Re: Maximum columns of a data table

Do you mean columns, or do you mean rows?

 

If you are thinking that you will run into an issue with too many columns, then I think your design is flawed.  You should not have this many columns.  I suspect that you will run into performance issues long before you start to hit the theoretical limits of the number of columns.

 

SQL itself supports 255 columns, so this will be the absolute upper limit.

ODBC / OLEDB have similar limits.

 

In regards to the number of rows, I believe the RowId value is a 32-bit Integer, so that puts a cap at something a little over 2 million.  But the Server Configuration query limits will also put a cap on it.

 

If you're getting towards the limits of the row count, then you likely also have a flawed design.  Geo SCADA's Data Table functionality isn't really intended to be a replacement for a stand alone database.  You will almost certainly suffer from performance limitations if trying to query for so many records from Geo SCADA.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..

See Answer In Context

Tags (1)
  • Tags:
  • english
Share
Reply
DavidSkilbeck
Lt. Commander DavidSkilbeck
Lt. Commander
‎2020-09-19 01:18 AM
0 Likes
5
794
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-19 01:18 AM

Re: Maximum columns of a data table

Hi,

thanks Bevan. Automation Interface externally and I am looking at approx. 70 columns to add.

I have added 5 test columns through code and was hoping I could create the rest in the next day or so.

Sounds like there should be no issues.

Thanks,

Tags (1)
  • Tags:
  • english
Share
Reply
BevanWeiss
Sisko BevanWeiss
Sisko
‎2020-09-20 04:34 AM
0 Likes
4
781
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-20 04:34 AM

Re: Maximum columns of a data table

70 columns does seem like a lot of data which is all strictly associated, and which you would expect to retrieve all at once.

I really can't think of a situation that would actually need 70 columns of data.

 

70 columns shouldn't be a problem for most of the data platforms you'd encounter, although possibly Excel CSV export might have problems with it, and you might start to push into untested territory around some other interfaces (like Crystal Reports, or the ClearSCADA ODBC driver etc).


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
Tags (1)
  • Tags:
  • english
Share
Reply
AdamWoodland
Commander AdamWoodland Commander
Commander
‎2020-09-22 03:09 PM
0 Likes
3
758
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-22 03:09 PM

Re: Maximum columns of a data table

Also the maximum recommended size of a data table is 100MB, and if its updated frequently it really shouldn't be anywhere near that size as the whole blob is synchronised each change.

Tags (13)
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA control
  • SCADA data
  • SCADA integration
  • SCADA network
  • SCADA programming
  • SCADA software
  • scada system
  • SCADA tutorial
  • SCADA web application
  • Telemetry and SCADA
Share
Reply
BevanWeiss
Sisko BevanWeiss
Sisko
‎2020-09-23 08:16 PM
0 Likes
2
748
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-23 08:16 PM

Re: Maximum columns of a data table

I did not know the whole blob would be sent for every change.

I really would have expected only 'transactions' to be sent.

 

I guess there's probably not much motivation to optimise this, it's a bit of a niche situation, and I wouldn't consider it 'recommended' usage of the Data Table items in Geo SCADA to be used for a volume of data approaching this (he says as he then thinks about those situations where he's probably got huge string tables being stored in Geo SCADA right now... like some alarm instruction database that allows for something like 4000 characters per alarm instruction, and many tens of thousands of alarm instructions)


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
Tags (13)
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA control
  • SCADA data
  • SCADA integration
  • SCADA network
  • SCADA programming
  • SCADA software
  • scada system
  • SCADA tutorial
  • SCADA web application
  • Telemetry and SCADA
Share
Reply
du5tin
Lieutenant JG du5tin
Lieutenant JG
‎2020-09-29 12:01 PM
0 Likes
1
702
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-09-29 12:01 PM

Re: Maximum columns of a data table

We use DataSets (different, but similar) with lots of columns to summarize data for oil and gas sites. Our master dataset has over 100 columns with various pieces of information possible at the different sites. The system has over 1,000 wells and maybe 25 major site types with lots of different meters and types of equipment. 

 

Because ClearSCADA JOIN performance is so terrible and types of joins are limited we decided to create this monster dataset instead. We were hitting major performance limitations joining two or more datasets together. If we re-architected with separate datasets and joins we could use more than 5 joins to get all the information the operations group is looking for.

 

It actually works, and works much better than doing joins. 

Tags (13)
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA control
  • SCADA data
  • SCADA integration
  • SCADA network
  • SCADA programming
  • SCADA software
  • scada system
  • SCADA tutorial
  • SCADA web application
  • Telemetry and SCADA
Share
Reply
BevanWeiss
Sisko BevanWeiss
Sisko
‎2020-10-01 08:44 PM
0 Likes
0
657
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content
‎2020-10-01 08:44 PM

Re: Maximum columns of a data table

That is definitely quite a wide dataset.

I'm surprised that there are 100 items of information per well though, even with a sewer pump station there's typically not that much meaningful information.

 

It is indeed challenging to get good performance out of Geo SCADA joins.  You really need to use a unique and indexed ID column to bridge the tables, and having that ID column index be cheap to compute is important also.

 

Things like FullName might be nice for us humans to reference, but they are expensive for the computer, even when indexed it still requires a hash calculation for every modification, and may still result in a partial linear search if there are hash collisions.

And having an ID column that is not explicitly unique and indexed is worthless.

 

I don't believe Data Sets themselves can have index columns, so joins will always exhibit performance challenges there 😞

I think the same situation applies for Data Grids.  I believe it is ONLY Data Tables that can have indexes.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
Tags (13)
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA control
  • SCADA data
  • SCADA integration
  • SCADA network
  • SCADA programming
  • SCADA software
  • scada system
  • SCADA tutorial
  • SCADA web application
  • Telemetry and SCADA
Share
Reply
Top Experts
User Count
sbeadle
Sisko sbeadle Sisko
188
BevanWeiss
Sisko BevanWeiss
56
AndrewScott
Lt. Commander AndrewScott
17
AdamWoodland
Commander AdamWoodland Commander
17
JChamberlain
Lt. Commander JChamberlain Lt. Commander
16
geoffpatton
Lt. Commander geoffpatton
10
See More Top Experts
Find a Service Provider
Find a certified partner to help you address your integration, installation, maintenance and project needs.
View all Providers
Support

Have a question? Please contact us with details, and we will respond.

Contact Us
FAQ

Look through existing questions to find popular answers.

Learn More
About

Want to know more about Exchange and its possibilities?

Learn More

Full access is just steps away!

Join Exchange for FREE and get unlimited access to our global community of experts.

Connect with Peers & Experts

Discuss challenges in energy and automation with 30,000+ experts and peers.

Get Support in Our Knowledge Base

Find answers in 10,000+ support articles to help solve your product and business challenges.

Ask Questions. Give Solutions

Find peer based solutions to your questions. Provide answers for fellow community members!

Register today for FREE

Register Now

Already have an account?Log in

About Us FAQ Terms & Conditions Privacy Notice Change your cookie settings
©2021, Schneider Electric