MySQLMove – An Easy To Use Tool To Move MySQL Database Objects and Data

Recently, I was asked to work on a project using a MySQL backend after many years of not having worked with the product. I was amazed when I saw that some of the more common tools to work with MySQL, such as MySQL Workbench or Toad, had still not developed an easy way to move objects and/or data between mySQL databases such as we have become accustomed to in SQL Server Management Studio.  Obviously, this greatly complicated my work and caused me to have to work with data in a production environment more often, which obviously I wasn’t comfortable with.

So like most developers, my brain began to think about how to solve this problem. You could of course, go through each object, get the CREATE script, put it in a giant file, and run it that way. That would be time consuming, bulky and difficult to maintain should there be changes to the database objects. And of course, this did nothing for the issue of data migration.

I looked at other products. MySQLDump, a free tool is fine. Of course, it was more complicated then I would have liked. I just want to move the objects and data and not have a high learning curve of a new product. Just get it done already. Others were available that you had to pay for. Being the cheap bastard that I am, that option didn’t appeal to me either.

So I wrote MySQLMove. This started out as a quick and dirty option to get objects and data moved to another database on another server and gradually moved to including things like a report, the option to only migrate objects, and actually put a user interface on top instead of just running a script. The script was fine for me but it may not be for other developers.

Let’s go into some things you need to know or generally be aware of.

First, some MySQL developers might wonder why I did not use INFILE to import data. The reason is for many such developers, they are coding against a shared hosting environment and often the administrators in such environment have disabled this option. Even using the LOCAL keyword presented problems relating to security. However, in future releases (but see below) I am planning on a interface modification that will allow the end user to indicate that INFILE is available for use and it will operate accordingly.

UPDATE:

The engine now uses INFILE and LOCAL keywords. It is also now lightening fast!

Please make sure you have created the database on the destination server. I know that comes from the “duh” department but you would be surprised…. If you are asking yourself the reason why I don’t do it for you, it is because I drop each object on the destination individually as it is being imported. But more importantly, often each database has a specific username and password designated for access. An idea submitted for future versions (but see below) is that we collect all databases under a given username and password and allow multiple databases to be migrated at one time.  That day is not here yet but it intrigues me.

Another “duh” is it is important to make sure the destination database doesn’t have any users writing changes.

Next…. it is important that if you are importing from or to a Linux based environment, that you are absolutely certain of the case of your database, its objects and server name. Windows is much more forgiving then Linux is in this regard. If you are importing from Linux, be prepared for windows to make all objects lower case, regardless of how you created them in Linux based servers. I have also seen some versions of MySQL run on Windows that were also sticklers for such. My best advice to you is to make all objects lower case. Both Linux and Windows based MySQL servers can handle such and if you are often moving between the two systems you will save yourself a lot of trouble.

Next, you will see an option in a checkbox to only import MySQL objects.

If you have the time and the data available to you in text or csv format, I suggest you check this. Why? While this engine can and will move data, it takes longer then a manual import would. The way it is built is all of your insert statements for the data are collected as one statement and inserted collectively. It was about 20 percent faster then inserting a row at a time and also if it should bomb, I can rollback the transaction so that you are not left with a table with partial data. That said, if you are one of those folks who build 300+ column tables, (by the way who hurt you as a child to do that to yourself? jk) it is going to take awhile. But rest assured, it will get there. This was tested against a table with 80000 rows and 30 columns. So it is pretty robust.

UPDATE:

As mentioned above I now use INFILE and LOCAL keyword. The above text is no longer a concern.

Next, it is important you let the code finish. Check that, it is imperative. If you are impatient, switch to decaf and try some breathing exercises.

Finally, MySQLMove will attempt to export database objects as follows: tables, stored procedures, functions and triggers. Not every single one of all those objects can be migrated using this tool. One issue that has commonly arisen is when the code uses a PREPARE statement along with a string for use as the command. Future versions (but see below) will probably rectify this. Should something not be exportable to the destination, at the end of the process a report will appear showing what objects weren’t able to be moved. In my testing, it is a rare occurrence. Also with triggers, remember the “defined user” is copied straight from the trigger to the destination. You will need to make sure that user exists on the destination database as well. Oh and by the way, triggers are migrated last. Otherwise each insert might cause that trigger to fire. And yeah that would not be fun.

Finally, I would love to continue developing this tool. There are so many things I could and would love to do with it. Better reports, table optimization, code optimization, object selection etc. Unfortunately that costs me time, which in translation for those that live under a rock, this means it costs me money. If you find the tool useful and would like to continue to see it developed, please don’t be a cheap bastard like me and make a donation here using my email address – kellyjmartens@hotmail.com .

 

I would be extremely grateful. I know the honor system is putting yourself out there isn’t real effective but it gives those of you who would like to say thank you and keep going with this a chance to do so.

So on to the download…. You have two options….

One if you are a person who already has the MySql.Data dll already installed on your system (make sure it is version 6.9.9.0 and you have at least .NET runtime v4.0.30319) you can download just the bin folder. If you don’t or aren’t sure, get the install package zip file called “MySqlMoveSetup”. If you need the .NET Framework you can download it here.

Both the bin folder zip file and the installer zip file are located here on my OneDrive.

Please do send me an email at kellyjmartens@hotmail.com with any bug reports, suggestions or praise. All are accepted.  

 

Have a great day!

Class to Simplify and Minimize Code for Stored Procedures and TSQL

First of all if you are using TSQL in your code you really need to get in a another line of work. It isn’t scalable, difficult to maintain and is generally a bad practice. But I know some of you still do it so I did include it. This class will allow you to in a few lines of code do your select, update, insert or delete statements in a very few lines of code. All that is required of the developer is to list the values in order (if you are using stored procedures) that are required. Also included is how to declare a new instance of a sqlparametercollection – which you aren’t suppose to be able to do. I ended up not needing it but included it just in case someone needs to do that someday. Also if you are using a strongly typed dataset there is a method included of how to do that. After the code is example of usage. Have a great day!

using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;

using System.IO;

using System.Data.SqlClient;

public class clsData
{

    public SqlConnection sqlconn = new SqlConnection();
    public string Error_Message;
    public List<SqlParameter> ParamList = new List<SqlParameter>();
    public ArrayList ParamValues = new ArrayList();

    public int SQLExecuteNonQueryValue;
    public clsData()
   

    public enum SQLAction
    {
        SelectAction,
        UpdateInsertDeleteAction
    }

    public virtual DataTable DataAction(clsData.SQLAction Action, DataTable dt, string cmdtext, string ParamValues = “”)
    {
        SQLExecuteNonQueryValue = -1;
        SqlCommand cmd = new SqlCommand();
        cmd.CommandText = cmdtext;
        cmd.CommandType = CommandType.StoredProcedure;
        cmd.Connection = sqlconn;
        this.GetParameterList(cmdtext);
        if (!string.IsNullOrEmpty(Error_Message)) {
            return null;
        }
        if (!string.IsNullOrEmpty(ParamValues)) {
            this.ParseSQLParameterValues(ParamValues);
            for (i = 0; i <= this.ParamList.Count – 1; i++) {
                cmd.Parameters.AddWithValue(this.ParamList[i].ToString(), this.ParamValues[i].ToString());
            }
        }
        //Debug.WriteLine(cmd.CommandText)
        try {
            cmd.Connection.Open();
            switch (Action) {
                case SQLAction.SelectAction:
                    dt.Load(cmd.ExecuteReader());
                    break;
                case SQLAction.UpdateInsertDeleteAction:
                    //Check this value to make sure everything went ok
                    SQLExecuteNonQueryValue = cmd.ExecuteNonQuery();
                    cmd.Connection.Close();
                    return null;
            }
            cmd.Connection.Close();

        } catch (Exception ex) {
            Error_Message = “There was a problem with ” + cmdtext + “. “;
            return null;
        }

        //removing duplicate table from dataset
        for (i = 0; i <= ds_Copy.Tables.Count – 1; i++) {
            if (ds_Copy.Tables(i).TableName == dt.TableName) {
                ds_Copy.Tables(i).Clear();
                ds_Copy.Tables(i).Columns.Clear();
            }
        }

        //placing datatable in the dataset
        ds_Copy.Merge(dt, false, MissingSchemaAction.Add);
        Error_Message = “”;
        return dt;
    }

    public DataTable DataActionTSQL(clsData.SQLAction Action, DataTable dt, string sqlstring)
    {
        SQLExecuteNonQueryValue = -1;
        SqlCommand cmd = new SqlCommand();
        cmd.CommandText = sqlstring;
        cmd.Connection = sqlconn;
        try {
            cmd.Connection.Open();
            switch (Action) {
                case SQLAction.SelectAction:
                    dt.Load(cmd.ExecuteReader());
                    break;
                case SQLAction.UpdateInsertDeleteAction:
                    //Check this value to make sure everything went ok
                    SQLExecuteNonQueryValue = cmd.ExecuteNonQuery();
                    cmd.Connection.Close();
                    return null;
            }
            cmd.Connection.Close();

        } catch (Exception ex) {
            Error_Message = “There was a problem with ” + sqlstring + “. “;
            return null;
        }
        //removing duplicate table from dataset
        for (i = 0; i <= ds_Copy.Tables.Count – 1; i++) {
            if (ds_Copy.Tables(i).TableName == dt.TableName) {
                ds_Copy.Tables(i).Clear();
                ds_Copy.Tables(i).Columns.Clear();
            }
        }
        //placing datatable in the dataset
        ds_Copy.Merge(dt, false, MissingSchemaAction.Add);
        Error_Message = “”;
        return dt;
    }

    private List<SqlParameter> GetParameterList(string ProcName)
    {
        //Get the parameters for the selected stored procedure
        List<SqlParameter> inputParamList = new List<SqlParameter>();
        ParamList.Clear();

        using (SqlConnection cn = new SqlConnection()) {
            DispatcherTool.My.MySettings Settings = new DispatcherTool.My.MySettings();
            cn.ConnectionString = Settings.SRAConnectionString;
            SqlCommand myCommand = new SqlCommand();
            myCommand.Connection = cn;
            myCommand.CommandText = ProcName;
            myCommand.CommandType = System.Data.CommandType.StoredProcedure;
            try {
                cn.Open();
                SqlCommandBuilder.DeriveParameters(myCommand);
                cn.Close();
            } catch (Exception ex) {
                Error_Message = “There was a problem with the connection to the database.”;
                return null;
            }

            //Dim sqlparams As SqlParameterCollection = DirectCast(GetType(SqlParameterCollection).GetConstructor(BindingFlags.NonPublic Or BindingFlags.Instance, Nothing, Type.EmptyTypes, Nothing).Invoke(Nothing), SqlParameterCollection)

            foreach (SqlParameter param in myCommand.Parameters) {
                if (param.Direction == System.Data.ParameterDirection.Input || param.Direction == System.Data.ParameterDirection.InputOutput) {
                    //sqlparams.Add(param.ParameterName & ” – ” & param.SqlDbType.ToString())
                    //Debug.WriteLine(param.ParameterName & ” – ” & param.SqlDbType.ToString())
                    inputParamList.Add(param);
                    //Else
                    // sqlparams.Add(param.ParameterName & ” – ” & param.SqlDbType)
                    //Debug.WriteLine(param.ParameterName & ” -2 ” & param.SqlDbType)
                }
            }
        }
        this.ParamList = inputParamList;
        Error_Message = “”;
        return this.ParamList;
    }

    public void ParseSQLParameterValues(string ValueString)
    {
        ParamValues.Clear();
        string[] parts = ValueString.Split(new char[] { ‘,’ });
        string part = null;
        foreach (string part_loopVariable in parts) {
            part = part_loopVariable;
            ParamValues.Add(part);
        }
    }

    //’Public Shared Function ConvertToTypedDataTable(Of T As {Data.DataTable, New})(ByVal dtBase As Data.DataTable) As T
    //’    Dim dtTyped As New T
    //’    dtTyped.Merge(dtBase)
    //’    Return dtTyped
    //’End Function

}

And then it’s use…..

private void Button1_Click(System.Object sender, System.EventArgs e)
{
    Strongly_Typed_DataSet.usp_Your_Stored_Procedure dt = new Strongly_Typed_DataSet.usp_Your_Stored_Procedure();

    DataGridView1.DataSource = cn.DataAction(clsData.SQLAction.SelectAction, dt, ds.usp_Get_PrinterName_for_PickList_Printing.ToString, “Tubing”);

    //DataGridView1.DataSource = cn.DataActionTSQL(clsData.SQLAction.SelectAction, dt, “SELECT *FROM Your_Table”)

}

Join me on Facebook

Fluent C# Review : Chapter One

I am reviewing an advanced copy of the book Fluent C# by noted .NET author Rebecca Riordan.

My focus today will be on Chapter One. Chapter One opens with an introduction to what application development is. It takes the reader through the simple problem of a messy paper stack and through a picture flow chart attempts to show the reader how the problem is to be solved and equate it with the same process of writing a windows application. It is unlike anything you have ever seen in a development book! But it is also one of the most clear renderings of the application process that I have seen.

They introduce you to the “clients” that are to appear throughout the book, two cooks named Neil and Gordon. (Incidentally the editor happens to be named Neil….coincidence? I think not!). It encourages you to take breaks at certain strategic points throughout the book as well. Neil and Gordon hand out their requirements and Riordan helps you get through it. She even begins the process of explaining the Agile software development methodology. It then introduces UML (Unified Modeling Language) and why it should be used. That and Agile are actually advanced topics perhaps left to a later time but I get why they do it here.

In the first project the book attempts to take you down the path of creating your requirements but before you see a single line of code you see the requirements explained in an easy to understand real life example. Then before you know it you have been introduced to database schemas, class diagrams and screen layout concepts all really done in a beautifully illustrated way. Extremely well done!

When there are words that need to be defined they are set out like so, so that they stand out from the rest of the page (see example below). Again hard to miss and it really catches the eye.

It goes into JIT (Just in Time Debugging) and the CLR (common language runtime) explaining what it is and how it is used. Of course at the end there is the obligatory review of the chapter. The only difference is this one is beautifully illustrated.

I don’t know about you but I am certainly looking forward to chapter two as we get more into the nuts and bolts of what is happening.

Check if SQL Database Exists with C#

Good Morning! I wanted to give a shout out to another blogger who linked to us yesterday here. It was so nice of him you know? Just amazing, the generosity of some folks out there!

Today’s code was a rewrite of the vb.net code here but using C#. One thing of note is that the Using keyword has a completely different meaning in C# so thus has been removed. It wasn’t really needed anyway.

Anyway, so make it a great day!

public Boolean testDatabaseExists(string server, string database)
        {
            String connString = (“Data Source=” + (server + “;Initial Catalog=master;Integrated Security=True;”));
            String cmdText = (“select * from master.dbo.sysdatabases where name=\'” + (database + “\'”));
            Boolean bRet;

            System.Data.SqlClient.SqlConnection sqlConnection = new System.Data.SqlClient.SqlConnection(connString);
            System.Data.SqlClient.SqlCommand sqlCmd = new System.Data.SqlClient.SqlCommand(cmdText, sqlConnection);
           
            try
            {
            sqlConnection.Open();
            System.Data.SqlClient.SqlDataReader reader = sqlCmd.ExecuteReader();
            bRet = reader.HasRows;
            sqlConnection.Close();
            }
            catch (Exception e)
            {
            bRet = false;
            sqlConnection.Close();
            MessageBox.Show(e.Message);
            return false;
            } //End Try Catch Block
                      
           
            if (bRet == true)
            {
                MessageBox.Show(“DATABASE EXISTS”);
                return true;
            }
            else
            {
                MessageBox.Show(“DATABASE DOES NOT EXIST”);
                return false;
            } //END OF IF

           
        } //END FUNCTION

Facebook