Using regular expression search/replace in Visual Studio

Note: I have transferred this blog from my old blog site: http://dcarapic.blogspot.com/ so I am reposting most of the stuff from there

In the previous post I have shown you how to do search/replace using .NET Regex. Visual Studio ‘Find and Replace’ dialogue (CTRL+H) also support regular expression search and replace and if you learn to use it (or better said, if you get into the habit of using it) you can do some pretty powerful stuff.

Visual Studio 2008 Search replace dialog

Note: Wherever you see me write ‘enter “xyz” into some text field’, it is assumed that you should enter it without the quotation marks
Note 2: Also all ‘questions’ here are simply a figment of my imagination, nobody asks me anything ๐Ÿ™

Q: I have pasted a bunch of code from the web into Visual Studio but I get double newlines, is there a way to strip them?
A: Why of course. Open the Find and replace dialogue. Enter \n+ into “Find what” field and \n into “Replace with” field (do not forget to check the “Use: Regular expressions”).
E: We say to VS: find any one or more continuous occurrences of newline character and replace it with just one newline character

The really interesting thing about regular expression search/replace is the usage of the grouping constructs to do a ‘capture’ and then use that capture in the replacement pattern. Great thing is that Visual Studio supports this! Unfortunately Visual Studio regular expression grouping constructs are not the same as .NET grouping constructs.
They are not difficult to learn but it is annoying that Microsoft did not settle for one standard (at least for them) way to use regular expression groupings. Instead of using parenthesis – (, ) to define capture groups you must use curly braces – {, } Instead of using dollar $ sign for using captures inside the replacement you must use the backslash \ character.

Q: I have a bunch of forms/web pages where I have methods where I first load data from an object to controls and then back from controls into an object. Do I have to type everything?
A: Of course not! We can copy the code we have and then use Find and replace to swap the assignment lines (something = somethingelse; into somethingelse = something;). For example we have the following method:

public static void FillControls(SomeObject o)
{
    txt1.Text = o.Property1;
    txt2.Text = o.Property2;
    txt3.Text = o.Property3;
    txt4.Text = o.Property4;
    txt5.Text = o.Property5;
    txt6.Text = o.Property6;
    txt7.Text = o.Property7;
    txt8.Text = o.Property8;
}

In order to create an ‘oposite’ method, we first make a copy of the first method and rename it:

public static void FillObject(SomeObject o)
{
    txt1.Text = o.Property1;
    txt2.Text = o.Property2;
    txt3.Text = o.Property3;
    txt4.Text = o.Property4;
    txt5.Text = o.Property5;
    txt6.Text = o.Property6;
    txt7.Text = o.Property7;
    txt8.Text = o.Property8;
}

Then we select the lines where we want to make a swap, hit CTRL+H to popup the Find and replace dialog, enter {.*} = {.*}; into the “Find what:” field, enter \2 = \1; into the “Replace with:” field, hit “Replace All” button and … :

public static void FillObject(SomeObject o)
        {
o.Property1 =             txt1.Text;
o.Property2 =             txt2.Text;
o.Property3 =             txt3.Text;
o.Property4 =             txt4.Text;
o.Property5 =             txt5.Text;
o.Property6 =             txt6.Text;
o.Property7 =             txt7.Text;
o.Property8 =             txt8.Text;
        }

E: We say to VS: find a group of characters before the = and assign it to a group (1), then find a group of characters after the = and assign it to a group (2); in the replacement first place the contents of the group (2), then the = and then contents of group (1)

We might experiment with the regular expression and get a better formatted version but it is easier to just hit the “Format the whole document” button (CTRL+E, D) which will format our code nicely.

Tagged with:

Using System.Text.RegularExpressions.Regex to do search/replace

Note: I have transferred this blog from my old blog site: http://dcarapic.blogspot.com/ so I am reposting most of the stuff from there

Google returns ~21.000.000 results (at the time of this writing) if you do a search for ‘regular expressions’ so you must be asking yourself why would somebody write something about it? Well sometimes you just need to use regular expressions to do some handy text processing, such as search/replace. I wrote this post just to give you some quick&dirty info on how to use the .NET Regex class to do substitutions.

First of, the simple search and replace. We search for a word and then replace it with another word:

public static void ReplaceSimple()
{
    string example = "I am a man.";
    string replaced = Regex.Replace(example, "man", "woman");
    Console.WriteLine(replaced);
    // Output: I am a woman.
    Console.ReadKey();
}

If you are using Regex for this kind of search/replace, then do yourself a favour and take a look at String.Replace method. Regular expressions get useful when you have some limitations on how you may do search and replace. Lets say that you wish to process some HTML (I am beating a dead cat, but who cares) and you would like to replace all <div> tags with <span> tags. One of the options is to use two replacements, one which replaces <div> with <span> and the other which replaces </div> with </span> (of course we could not just replace “div” with “span” because “div” might appear as a part of a HTML body text). But, doing it that way is not so interesting (and also makes this post pretty useless), so lets do something complex:

public static void ReplaceDiv()
{
    string example = "<div>Becomes span</div>";
    string replaced = Regex.Replace(example, @"<(/{0,1})div>", @"<$1span>");
    Console.WriteLine(replaced);
    // Output: <span>Becomes span</span>
    Console.ReadKey();
}

The secret to doing some complex search/replace in .NET (and I guess in many other regular expression implementations) is to define regular expression ‘capture’ groups inside the search pattern and then use them inside the replacement pattern. In the code above we are saying “give me all matches that start with <, have 0 or 1 / and end with div>; also group 0 or 1 /; then replace the found matches with <, followed by the first group and then followed by span>. The parenthesis (, ) characters serve as grouping constructs which we are using to ‘capture’ an appearance of 0 or 1 / character.

Any time that we use the parenthesis we are creating a group ‘capture’ which can be used in the replacement pattern by using the dollar ($) character. Using parenthesis for capture groups is simple but may lead to regular expression search patterns which are not immediately obvious (as if any useful regular expression patterns are obvious :)). To make it easier you may use named grouping constructs:

public static void ReplaceDivNamedGroup()
{
    string example = "<div>Becomes span</div>";
    string replaced = Regex.Replace(example, @"<(?<slash>/{0,1})div>", @"<${slash}span>");
    Console.WriteLine(replaced);
    // Output: <span>Becomes span</span>
    Console.ReadKey();
}

Naming the grouping is simple, just add ?<name> after the parenthesis start. You may then use the name of that group with syntax ${name} inside the replacement pattern.

As a bizarre fact, you may use the grouping constructs inside the search pattern. Example:

public static void EhThatRegex()
{
    string example = "We are searching for Yoda speak, searching are we";
    var match = Regex.Match(example, @"(\w+)\s+(\w+)\s+.*\2\s+\1");
    Console.WriteLine(match.Success );
    // Output: true
    Console.ReadKey();
}

Here we go, I’ve posted a non-obvious regular expression. But it can not be helped. The grouping ‘capture’ can be used inside the search pattern, we just may not use the $1 or ${xxx} but rather \1 and \k<xxx>. Of the top of my head I can not think where such a pattern would be useful but you might.

Here is the explanation of the search: (\w+)\s+(\w+)\s+.*\2\s+\1 search for a sequence of characters ending with space character (\w+)\s+ (basically we are searching for a complete word) and capture it in group 1; search again for another word and capture it in group 2; skip anything until you reach a point where the second word, followed by the first word appears. I am sure that some regular expression guru could use this in some way to check if the sentence is a palindrome, but that is beyond my abilities.

Tagged with: ,

Creating semi-automated database update scripts for Oracle

Note: I have transferred this blog from my old blog site: http://dcarapic.blogspot.com/ so I am reposting most of the stuff from there

If you have ever created applications that use a database to store data you must have run into a problem how to do updates on the database once the application goes into production and you need to update the production database structure and/or data with new functionality or bug fixes made on your development database. Some database tools (Oracle SQL developer) help you in this by having an integration with SVN. This means that you have a history of various database versions that you have been working on, but it takes some work to use that data to create update scripts which you can apply on your production system.

If you are using Oracle as your database solution then this problem becomes much easier because Oracle supports DDL triggers. DDL triggers enable you to do some work once a DDL command is executed. Combining this trigger with some logging can enable you to quickly produce database update scripts. Note: Everything done on the “SYS” schema is done as a SYS user. Everything else is done by ‘normal’ db users.To start with, we are going to need a database table to log our changes:

CREATE TABLE "SYS"."DDL_LOG" (   
    "ID" NUMBER(38,0) DEFAULT 0 NOT NULL ENABLE,
    "USER_NAME" VARCHAR2(30 BYTE),
    "DDL_DATE" DATE,
    "DDL_TYPE" VARCHAR2(30 BYTE),
    "OBJECT_TYPE" VARCHAR2(18 BYTE),
    "OWNER" VARCHAR2(30 BYTE),
    "OBJECT_NAME" VARCHAR2(128 BYTE),
    "SQL_TEXT" CLOB,
    "EXECUTE_TEXT" CLOB
);

Here is the explanation of the columns:

  • ID – id of the log (I’ll explain later why we need it)
  • USER_NAME – name of the Oracle user that made the DDL command
  • DDL_DATE – date and time of the modification
  • DDL_TYPE – type of modification (ALTER, CREATE etc.)
  • OBJECT_TYPE – type of object on which the modification was made (TABLE, TRIGGER etc.)
  • OWNER – schema owner
  • OBJECT_NAME – name of the object that was modified (created, altered dropped etc.)
  • SQL_TEXT- actual command that was executed
  • UPDATE_TEXT -I’ll explain this later

Here is the actual trigger that will do the logging:

CREATE OR REPLACE TRIGGER "SYS"."DDL_TRIG"
    AFTER DDL
ON DATABASE
DECLARE
    v_date varchar2(100);
    v_id number;
    v_stmt clob;
BEGIN
    IF ora_dict_obj_owner = 'DEVELOPMENT' THEN
        v_date := TO_CHAR(sysdate, 'YYYY-MM-DD');
        SELECT ddl_log_seq.nextval INTO v_id FROM dual;
        v_stmt := REPLACE(last_sql_text(), '''', '''''');

       INSERT INTO DDL_LOG (
        id,
        user_name,
        ddl_date,
        ddl_type,
        object_type,
        owner,
        object_name,
        sql_text,
        execute_text
        ) VALUES (
        v_id,
        ora_login_user,
        SYSDATE,
        ora_sysevent,
        ora_dict_obj_type,
        ora_dict_obj_owner,
        ora_dict_obj_name,
        last_sql_text(),
        'execute_if_not_executed(''' || ora_login_user || ''', ''' || v_date
        || ''', ' || v_id || ', ''' || v_stmt || ''');'
        );
    END IF;
END;

The trigger is pretty straightforward, it just fills the DDL_LOG table if the DDL commands were made on the DEVELOPMENT schema. What is interesting are the fields ID and EXECUTE_TEXT.

Why do we need ID field? This is because, later on, we need to create some sort of a script from the records in the DDL_LOG table. This script should have all the commands executed in the same order as the developers have executed them on the development database. While we could use DDL_DATE field for this, it is not a good solution when you are using database management tools.

The tools (such as PL/SQL developer or Oracle SQL developer) usually execute DDL commands in a batch so you get a lot of commands within the same millisecond. Ordering by DDL_DATE field does not help you much in those cases, but if you have ID then you can be certain of the order in which the DDL commands were executed.

UPDATE_TEXT field also serves to help us create an update script. This field will be filled with a procedure call (‘execute_if_not_executed’) that can be executed on the production server. Why would we need such an elaborate system? Why not just take the SQL_TEXT and run it on the production? This is certainly possible, but one of the things that might occur (and does occur) on the production database (and even more so if you have multiple production databases) is that a DDL command or a group of commands never get executed!

You may simply overlook a system because you forgot about it, or the guy responsible for updates got sick and someone else does a bad job out of it. If such an error occurs frequently it would be nice to have a log of updates (DDL commands) that have been made on the database. It would be even nicer if you could automatically check if an update was executed and execute it if necessary. This is exactly what the ‘execute_if_not_executed’ function does: Checks if we have already executed a command If not then executes it Records that the update has been made so that future calls will not do an update again The update records are kept in a database table for each production database.

CREATE TABLE "PRODUCTION"."SCHEMA_VERSION" (
    "VERSION_USER" VARCHAR2(50 BYTE) NOT NULL ENABLE,
    "VERSION_DATE" DATE NOT NULL ENABLE,
    "VERSION_NUMBER" NUMBER NOT NULL ENABLE,
    CONSTRAINT "SCHEMA_VERSION_UK" UNIQUE
    ("VERSION_USER", "VERSION_DATE", "VERSION_NUMBER") ENABLE
);

Here is the EXECUTE_IF_NOT_EXECUTED procedure.

CREATE OR REPLACE PROCEDURE "PRODUCTION"."EXECUTE_IF_NOT_EXECUTED"
(
    p_user in VARCHAR2,
    p_date in VARCHAR2,
    p_order_no in NUMBER,
    p_command in VARCHAR2
)
    AUTHID CURRENT_USER
AS
BEGIN
    IF EXISTS_VERSION(p_user, p_date, p_order_no) = 0 then
        EXECUTE IMMEDIATE p_command;

        insert_version(p_user, p_date, p_order_no);
        commit;
    end if;
END;

The EXECUTE_IF_NOT_EXECUTED procedure is relatively simple. It takes parameters which serve to uniquely identify the update command (user, date and id) and the actual DDL command text. One thing that is specific is that the procedure has AUTHID CURRENT_USER option set. This gives the procedure will execute the DDL commands with the same rights as the user that is executing the procedure. You may remove this option which means that the rights of the user that created the procedure will be used when executing the DDL commands.
This depends on how you have set up your update process. Once you have everything in place, all DDL commands that you are making on developer database will be recorded into the DDL_LOG table. When you want to create an update script just browse the table and copy/paste the update commands into a file, surround them with begin/end; and you are good to go.

Appendix: Several various functions and procedures have also been used but I will not explain them since they are pretty much straightforward. I am just going to list them here.

CREATE OR REPLACE FUNCTION "PRODUCTION"."LAST_SQL_TEXT"
    RETURN CLOB IS
    v_sql_text ORA_NAME_LIST_T;
    v_stmt CLOB;
    n INTEGER;
    i INTEGER;
BEGIN
    n := ORA_SQL_TXT(v_sql_text);
    FOR i IN 1..n LOOP
        v_stmt := v_stmt || REPLACE(SQL_TEXT(i), chr(0));
    END LOOP;
    RETURN v_stmt;
END;

CREATE OR REPLACE FUNCTION "PRODUCTION"."EXISTS_VERSION"
(
    p_username in VARCHAR2,
    p_isoDate in VARCHAR2,
    p_versionNum in NUMBER
)
    RETURN NUMBER
    IS
    v_counter NUMBER;
BEGIN
    SELECT COUNT(1) INTO v_counter FROM SCHEMA_VERSION
    WHERE VERSION_USER = p_username AND
    VERSION_DATE = parse_iso_date(p_isoDate) AND
    VERSION_NUMBER = p_versionNum;
    IF counter = 0 then
        RETURN 0;
    ELSE
        RETURN 1;
    END IF;
END;

CREATE OR REPLACE PROCEDURE "PRODUCTION"."INSERT_VERSION"
(
    p_username in VARCHAR2,
    p_isoDate in VARCHAR2,
    p_versionNum in NUMBER
)
    IS
BEGIN
    INSERT INTO SCHEMA_VERSION (VERSION_USER, VERSION_DATE, VERSION_NUMBER)
    values (p_username, parse_iso_date(p_isoDate), p_versionNum);
END;


CREATE OR REPLACE FUNCTION "PRODUCTION"."PARSE_ISO_DATE"
(
    p_isoDate in varchar2
)
    RETURN DATE
    IS
BEGIN
    RETURN TO_DATE(p_isoDate, 'YYYY-MM-DD');
END;
Tagged with:

Using “hole in the middle” pattern with methods to insure pre and post processing

Note: I have transferred this blog from my old blog site: http://dcarapic.blogspot.com/ so I am reposting most of the stuff from there

Sometimes it is important to do processing before and after executing a piece of code (or a function). This is usually necessary when you have unmanaged components that you wish to use. C# solves this by using the using keyword. Example of this would be the various ASP.NET connections.

var connectionString = GetConnectionString();
using (var adapter = new SqlConnection(connectionString))
{
    // do something
}

While using using is a good solution it does require the programmer to be disciplined and not to forget it. Another drawback of this solution is that requires that you have a component which implements IDisposable and you must always instantiate it in order to use it.
Another possible solution would be to use “hole in the middle pattern” to achieve the similar functionality. In short “hole in the middle” refers to having some sort of activity/processing where a part of processing is provided by an external source and this is exactly what we need.

We need to ensure that we will execute some code before and after the ‘main’ code gets executed. Here is how we could organize the code so that we ensure that SqlConnection gets properly disposed

public static class ContinuationTest1
{
    public static void ExecuteSql(Action methodToExecute)
    {
        var connectionString = GetConnectionString();
        using (var adapter = new SqlConnection(connectionString))
        {
            methodToExecute(adapter);
        }
    }
}

Using this new method is relatively simple

public void UseContinuationTest1()
{
    ContinuationTest1.ExecuteSql((con) =&gt;
    {
        con.CreateCommand();
        // do something
    });
}

Note that I am using new Lambda style method definitions as it makes this sort of programming style really easy to use. Here is another example of how we could achieve exception wrapping

public static void ExceptionWrap(Action methodToExecute)
{
    try
    {
        methodToExecute();
    }
    catch (Exception ex)
    {
        var exceptionHandler = GetExceptionHandler();
        exceptionHandler.Handle(ex);
    }
}

Using it is, again, simple

public void UseContinuationTest2()
{
    ContinuationTest2.ExceptionWrap(() =&gt;
    {
        // do something that might throw an exception
    });
}

Another advantage that we get from this is that we can hide the creation of parameters that we provide to our “method in the middle”. This could be used as (simple) (or with) Dependency Injection mechanism where we want to ensure a consistent way to access components or services in our code. Here is a variant of the first example where we do not provide the SqlConnection but rather an interface (IDbConnection). I have also added a variant where we can ensure that everything is done inside a transaction.

public static class ContinuationTest3
{
    public static void ExecuteSql(Action methodToExecute)
    {
        var connectionString = GetConnectionString();
        using (var adapter = new SqlConnection(connectionString))
        {
            methodToExecute(adapter);
        }
    }
    public static void ExecuteSqlInTransaction(Action methodToExecute)
    {
        using (var ts = new TransactionScope())
        {
            ExecuteWithAdapter(methodToExecute);
            ts.Complete();
        }
    }
}

And using it

public void UseContinuationTest3()
{
    ContinuationTest3.ExecuteSqlInTransaction((con) =&gt;
    {
        con.CreateCommand();
        // do something, transaction will be automaticaly commited
    });
}

While this kind of programming looks great, there are some drawbacks to it if you are using the Lambda style methods to define the “middle method” that will get executed. Drawbacks with Lambdas are: Uglier stack trace – Lambda is just a nicer way to define an inline method. It is still a method and the C# compiler will give it an ugly name Method inside a method – This might be the a subjective issue to some. Wherever you are using this style you will need to have two methods, the outside method and the “middle” method.
Using Lambdas makes the “middle” method look like it is not there, but it still is. Variable in outer scope – Make sure that you understand what happens if you use an outer scope variable inside the lambda method. This is something that is good to know in general ๐Ÿ™‚ I hope that you can see that the power that we can get from this style of programming.

Tagged with: , , ,

Welcome to my blog

Note: I have transferred this blog from my old blog site: http://dcarapic.blogspot.com/ so I am reposting most of the stuff from there

Disclaimer (are we legal or what): English is not my native language so please do not flame (too much) if my sentences are not constructed as they should be.

Now to the real stuff: Hello to anybody who might be reading this (I guess nobody at the time I am writing this, but who knows, I might get a horde of followers in the future :)) I have been programming for some time now, several years as an amateur and several years as a paid “professional”. I have started with C++ and moved on to .NET and C# when it reached 1.1. I have always wanted to share my knowledge with the programming community. The primary reason for that is that I have gained a lot of knowledge following various blogs and programming portals and I felt the need to contribute to that community.

I have opened my original blogs some four years ago and I have always pushed back my first blog since I always felt that I am not at that ‘level’ where my thoughts can help somebody. After all this time I realized that I will probably never be satisfied with me enough so that I can boldly blog about some subject.

So I will just blog and let whoever is reading this decide if what I write is good enough.

Top